-- Logs begin at Tue 2022-08-30 23:32:40 UTC, end at Wed 2022-08-31 01:14:07 UTC. -- Aug 31 00:06:16.723726 dv-cinder-855060-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 31 00:06:16.732705 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 31 00:06:16.736483 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 31 00:06:16 2022] *** Aug 31 00:06:16.736483 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 31 00:06:16.736483 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 31 00:06:16.736483 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: nodename: dv-cinder-855060-1-iscsi Aug 31 00:06:16.736483 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: machine: x86_64 Aug 31 00:06:16.736483 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: clock source: unix Aug 31 00:06:16.736483 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: pcre jit disabled Aug 31 00:06:16.736483 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: detected number of CPU cores: 4 Aug 31 00:06:16.736483 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: current working directory: / Aug 31 00:06:16.736483 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: detected binary path: /usr/bin/uwsgi-core Aug 31 00:06:16.736483 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: your processes number limit is 47680 Aug 31 00:06:16.736483 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: your memory page size is 4096 bytes Aug 31 00:06:16.736483 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: detected max file descriptor number: 2048 Aug 31 00:06:16.736483 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: lock engine: pthread robust mutexes Aug 31 00:06:16.737440 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: thunder lock: enabled Aug 31 00:06:16.737440 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 31 00:06:16.737440 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 31 00:06:16.776040 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: Python main interpreter initialized at 0x55f5bfd949c0 Aug 31 00:06:16.776040 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: python threads support enabled Aug 31 00:06:16.776040 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: your server socket listen backlog is limited to 100 connections Aug 31 00:06:16.776040 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: your mercy for graceful operations on workers is 90 seconds Aug 31 00:06:16.776388 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: mapped 403077 bytes (393 KB) for 2 cores Aug 31 00:06:16.776453 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: *** Operational MODE: preforking *** Aug 31 00:06:16.776724 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: *** uWSGI is running in multiple interpreter mode *** Aug 31 00:06:16.776724 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: spawned uWSGI master process (pid: 81458) Aug 31 00:06:16.776933 dv-cinder-855060-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 31 00:06:16.777976 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: spawned uWSGI worker 1 (pid: 81471, cores: 1) Aug 31 00:06:16.777976 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: spawned uWSGI worker 2 (pid: 81472, cores: 1) Aug 31 00:06:16.777976 dv-cinder-855060-1-iscsi devstack@keystone.service[81458]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 31 00:06:21.140905 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 00:06:21.140905 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: from cryptography.utils import int_from_bytes Aug 31 00:06:21.141746 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 00:06:21.141746 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: from cryptography.utils import int_from_bytes Aug 31 00:06:21.142600 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 00:06:21.142600 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: from cryptography.utils import int_from_bytes Aug 31 00:06:21.143083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 00:06:21.143083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: from cryptography.utils import int_from_bytes Aug 31 00:06:21.192988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 00:06:21.193188 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 00:06:21.193363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] command line args: [] {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 00:06:21.193640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 00:06:21.193825 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ================================================================================ {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 00:06:21.194079 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] admin_token = **** {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.194268 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 00:06:21.194357 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] config_dir = [] {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.194465 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 00:06:21.194555 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.194661 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] command line args: [] {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 00:06:21.194747 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] config_source = [] {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.194823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] debug = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.194932 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 00:06:21.195021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.195281 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ================================================================================ {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 00:06:21.195422 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] admin_token = **** {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.195500 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.195566 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.195633 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] config_dir = [] {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.195799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.195882 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.195987 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] config_source = [] {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.196071 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.196137 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] debug = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.196265 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] list_limit = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.196349 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.196438 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] log_config_append = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.196577 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.196685 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.196795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.196898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] log_dir = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.197000 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] log_file = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.197094 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.197201 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.197312 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.197399 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.197518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] list_limit = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.197621 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] log_config_append = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.197698 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.197820 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.197902 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.197987 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] log_dir = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.198103 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.198181 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] log_file = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.198301 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.198384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.198466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.198584 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.198669 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.198831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.198915 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.199097 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.199236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.199343 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.199467 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.199580 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.199693 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.199805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.199925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.200039 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.200152 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.200264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.200382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.200502 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.200635 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.200749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.67/identity {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.200869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.200981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] publish_errors = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.201105 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.201220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.201331 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.201449 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.201568 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.201682 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.201801 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.201996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.202114 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.67/identity {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.202266 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] publish_errors = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.202389 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] standard_threads = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.202503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.202632 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.202746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.202868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.202982 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.203177 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.203177 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] use_journal = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.203403 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.203516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] use_json = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.203676 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.204425 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] use_stderr = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.204562 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] standard_threads = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.204678 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] use_syslog = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.204776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.204889 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.205010 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.205125 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.205276 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.205276 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] use_journal = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.205495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.205607 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] use_json = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.205726 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.205847 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] use_stderr = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.205847 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] use_syslog = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.206070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.206190 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 00:06:21.206311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.206439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.206560 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.206675 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.206787 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.206787 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.206996 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.206996 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.207239 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.207359 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.207476 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.207590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.207708 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.207830 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.207944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.208056 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.208175 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.208175 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.208401 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.208523 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.208637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.208750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.208863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.208863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.209085 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.209205 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.209319 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.209433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.209551 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.209671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.209784 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.209970 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.210086 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.210199 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.210312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.210424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.210544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.210669 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.210793 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.210907 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.211020 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.211150 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.211263 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.211393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.211511 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.211511 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.211721 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.211833 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.211953 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.212067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.212186 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.212186 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.212398 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.212517 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.212639 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.212639 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.212852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.212852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.213060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.213179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.213300 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.213300 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.213517 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.213517 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.213735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.213855 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.213977 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.213977 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.214201 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.214315 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.214436 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.214557 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.214679 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.214794 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.214906 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.215019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.215193 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.215309 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.215496 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.215613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.215733 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.215847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.215960 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.216079 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.216200 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.216315 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.216427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.216540 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.216652 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.216765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.216885 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.216999 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.217112 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.217225 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.217338 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.217450 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.217570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.217685 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.217799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.217918 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.218032 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.218145 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.218250 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.218346 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.218440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.218535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.218630 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.218754 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.218863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.218961 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.219082 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.219194 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.219262 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.219872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.219995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.220101 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.220203 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.220203 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.220406 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.220525 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.220648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.220648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.220952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.221083 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.221200 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.221320 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.221426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.221547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.221672 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.221787 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.221900 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.221900 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.222103 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.222222 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.222344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.222448 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.222560 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.222560 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.222787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.222913 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.223071 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.223189 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.223303 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.223416 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.223529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.223654 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.223770 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.223873 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.223985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.224099 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.224213 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.224325 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.224443 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.224547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.224659 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.224772 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.224886 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.225007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.225124 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.225237 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.225351 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.225463 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.225577 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.225698 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.225814 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.225932 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.226047 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.226161 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.226274 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.226394 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.226510 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.226623 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.226735 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.226848 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.226961 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.227163 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.227291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.227406 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.227520 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.227633 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.227747 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.227868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.227992 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.228122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.228238 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.228352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.228466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.228585 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.228700 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.228821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.228943 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.229057 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.229171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.229284 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.229407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.229523 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.229645 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.229751 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.229870 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.229986 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.230109 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.230230 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.230353 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.230469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.230583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.230696 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] auth.external = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.230818 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.230932 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.231093 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.231241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.231372 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.231489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.231604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.231718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] auth.password = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.231841 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.231957 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] auth.token = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.232072 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] auth.external = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.232186 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.232301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.232416 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.232540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.232657 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.232771 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.232885 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.233006 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] auth.password = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.233122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.233249 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] auth.token = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.233365 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.233480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.233595 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.233711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.233825 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] credential.caching = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.233947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.234064 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.234179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.234293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.234411 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.234534 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.234921 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.235596 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.235720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.235835 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.235949 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] credential.caching = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.236077 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.236251 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.236410 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.236528 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.236644 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.236758 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.236872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.236994 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.236994 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.237209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.237324 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.237324 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.237543 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.237670 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.237787 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.237900 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.238014 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.238128 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.238241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.238363 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.238479 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.238599 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.238714 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] federation.caching = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.238829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.238950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.239092 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.239210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.239324 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.239438 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.239552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] federation.caching = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.239665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.239787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.239904 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.240018 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.240132 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.240246 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.240359 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.240481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.240596 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.240710 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.240833 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.240954 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.241081 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.241197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.241311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.241424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.241538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] identity.caching = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.241651 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.241847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.241966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.242080 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.242194 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.242308 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.242423 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] identity.caching = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.242544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.242666 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.242781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.242895 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.243009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.243208 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.243342 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.243459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.243574 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.243689 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.243803 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.243917 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.244041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.244157 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.244271 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.244271 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.244489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.244604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.244729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.244845 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.244960 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.245075 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.245189 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.245303 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.245425 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.245541 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.245656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.245771 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.245885 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.245999 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.246123 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.246239 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.246353 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.246468 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.246582 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.246696 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.246819 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.246935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.247067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.247185 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.247299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.247414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.247544 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.247662 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.247776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.247890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.248004 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.248153 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.248283 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.248400 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.248515 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.248664 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.248788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.248905 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.249027 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.249144 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.249260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.249374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.249488 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.249602 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.249724 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.249840 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.249961 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.250077 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.250191 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.250307 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.250430 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.250552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.250684 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.250817 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.250937 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.251117 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.251252 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.251355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.251470 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.251585 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.251701 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.251815 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.251939 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.252055 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.252169 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.252283 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.252648 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.252765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.252888 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.253012 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.253129 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.253243 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.253357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.253481 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.253598 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.253713 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.253827 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.253941 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.user = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.254055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.254178 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.254296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.254414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.254530 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.254644 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.254758 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.254880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.254996 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.user = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.255129 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.255244 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.255359 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.255483 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.255609 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.255725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.255840 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.255955 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.256069 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.256184 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.256306 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.256422 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.256536 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.256651 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.256765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.256879 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.257001 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.257117 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.257232 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.257346 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.257460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.257574 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.257698 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.257815 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.257929 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.258044 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.258157 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.258272 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.258394 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.258510 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.258625 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.258741 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.258855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.258969 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.259140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.259259 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.259373 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.259487 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.259602 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.259717 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.259839 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.259955 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.260070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.260184 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.260298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.260412 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.260534 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.260650 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.260766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.260880 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.260994 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.261108 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.261230 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.261346 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.261466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.261583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.261697 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.261812 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.261933 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.262049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.262163 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.262277 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.262392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.262507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.262629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] resource.caching = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.262745 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.262860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.262974 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.263102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.263218 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.263344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.263461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.263575 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.263689 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] resource.caching = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.263803 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.263919 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.264041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.264158 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.264272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.264391 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.264505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.264619 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.264807 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.264925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.265041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] role.caching = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.265162 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.265278 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] role.driver = sql {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.265393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.265516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.265632 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.265747 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.265861 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.265975 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.266092 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] role.caching = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.266225 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.266358 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] role.driver = sql {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.266474 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.266590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.266705 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.266980 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.267150 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.267267 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.267382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.267497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.267611 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.267738 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.267855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.267970 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.268091 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.268209 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.268209 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.268431 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.268548 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.268663 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.268778 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.268892 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.269007 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.269129 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.269246 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.269361 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.269476 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.269597 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.269713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.269835 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.269952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.270066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.270180 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.270295 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.270410 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.270533 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.270649 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.270764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.270878 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.270992 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.271186 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.271316 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.271433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.271547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.271661 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.271775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.271897 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.272013 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.272127 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.272242 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.272357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.272470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.272592 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.272708 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.272823 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.272936 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.273050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.273164 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.273285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.273403 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.273523 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.273639 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] token.caching = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.273753 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.273914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.274041 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.274158 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.274307 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.274454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.274572 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] token.caching = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.274686 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.274813 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.274968 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.275100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.275216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.275330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.275445 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.275573 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.275690 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.275804 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.275918 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.276033 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.276146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.276269 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.276385 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.276501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.276615 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.276729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.276872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.277001 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.277118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.277233 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.277346 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.277471 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.277596 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.277713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.277834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.277951 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.278065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] database.connection = **** {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.278179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.278302 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.278426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.278542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.278658 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.278780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.278897 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] database.connection = **** {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.279010 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.279135 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.279255 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.279369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.279495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.279612 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.279726 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.279840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.279954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.280068 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.280191 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.280306 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.280421 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.280542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.280657 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.280773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.280895 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.281011 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.281125 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.281239 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.281352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.281466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.281588 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.281704 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.282325 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.282447 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.282665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.282784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.282908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.282908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81471) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 00:06:21.283161 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.283161 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.283161 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.283161 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 00:06:21.283161 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81472) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 00:06:21.310847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.311103 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.311329 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.311861 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.312095 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.312322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.319919 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.320804 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.348208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.348445 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.349487 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.349722 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.402516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.403214 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.451831 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.452067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.452549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.452790 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.608776 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.609019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.609242 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.609445 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.609535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.609666 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.609748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.610404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.610496 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.610691 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.611123 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.611382 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.634286 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.634524 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.634616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.634802 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.634900 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.635050 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.635139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.635277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.635362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.635496 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.635580 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.635717 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.635800 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.635933 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.647349 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.647998 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.660334 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.660568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.660839 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.660977 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.661062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.661214 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.661298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.661444 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.661527 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.661666 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.661750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81471) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.661882 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.662096 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.662310 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81472) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 00:06:21.679415 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.679627 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 00:06:21.679972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.680057 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.680177 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 00:06:21.680261 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 00:06:21.680456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.680582 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.680724 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.680808 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 00:06:21.680987 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.681078 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.681250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.681341 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.681802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.682068 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.690975 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.691274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.691547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.691837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.691922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.692192 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.692467 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.692758 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.698850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.699937 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.701760 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.702046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.702330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.702612 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.702842 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.702924 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.703158 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.703244 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.703455 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.703549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.704245 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.704355 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.704559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.704650 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.704897 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.704985 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.705169 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.705298 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.705581 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.705980 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.706265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.706544 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.728554 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.729952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.730056 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.730289 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.731455 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.731744 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.734823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.735183 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.735470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.736582 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.736869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.737154 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.741943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.742236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.743562 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.743855 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.746195 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.746543 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.746854 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.747174 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.747458 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.748071 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.748433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.748734 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.749040 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.749321 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.749421 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.749608 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.749702 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.749881 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.749974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.750150 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.765911 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.766736 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.768686 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.768971 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.770706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.772901 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.773839 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.774138 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.776173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.776474 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.778504 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.780762 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.781699 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.784628 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.786910 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.787907 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.788158 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.788268 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.788481 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.788579 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.788764 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.788848 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.789057 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.789179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.789338 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.789429 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.806080 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.806369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.806640 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.806909 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.808454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.808750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.809031 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.809297 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.814487 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.816045 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.817249 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.817594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.817876 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.818899 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.820468 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.820554 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.820748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.820834 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.821209 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.821498 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.821801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.823400 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.823692 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.823977 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.824260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.824554 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.833265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.833552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.833938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.834227 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.834514 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.834794 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.835104 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.836032 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.836322 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.836605 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.836888 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.837171 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.837453 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.837748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.851362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.853771 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.854282 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.856816 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.857211 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.858658 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.859783 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.860904 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.861183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.861468 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.861799 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.863296 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.863582 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.863867 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.867967 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.869457 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.870635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.872137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.872504 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.872806 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.873092 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.873371 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.875018 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.875346 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.876195 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.876528 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.882133 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.882428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.882713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.884743 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.885077 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.885319 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.889696 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81472) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.889985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.890272 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.890556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.890834 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.891164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.892099 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.892389 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.892666 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.892864 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81471) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 00:06:21.892963 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.893149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.893242 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.893441 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.893572 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.893720 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.893804 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.893995 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.894077 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81472) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.894269 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.894557 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.894833 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.895146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.895429 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.895703 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.895967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.896235 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.896511 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81471) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 00:06:21.929358 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f5bfd949c0 pid: 81472 (default app) Aug 31 00:06:21.931870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f5bfd949c0 pid: 81471 (default app) Aug 31 00:06:21.932870 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce030357-f0b3-46c0-9ec9-46926ac5b2ec None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:21.933162 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce030357-f0b3-46c0-9ec9-46926ac5b2ec None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:21.933438 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce030357-f0b3-46c0-9ec9-46926ac5b2ec None None] PATH_INFO: `/v3` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:21.937621 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1/1] 10.222.0.67 () {50 vars in 791 bytes} [Wed Aug 31 00:06:21 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:28.603714 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef20403-87ee-4334-9a80-413ed30f31ae None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:28.604331 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef20403-87ee-4334-9a80-413ed30f31ae None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:28.604331 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef20403-87ee-4334-9a80-413ed30f31ae None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:28.608508 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1/2] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:06:28.614316 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c942be-2ecd-493a-b874-09b6b3e4785b None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:28.614872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c942be-2ecd-493a-b874-09b6b3e4785b None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:28.614872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c942be-2ecd-493a-b874-09b6b3e4785b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:28.686343 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: INFO dbcounter [None req-56c942be-2ecd-493a-b874-09b6b3e4785b None None] Registered counter for database keystone Aug 31 00:06:28.705497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG oslo_db.sqlalchemy.engines [None req-56c942be-2ecd-493a-b874-09b6b3e4785b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81472) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 00:06:28.711761 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writer thread running {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 00:06:28.894104 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG passlib.handlers.bcrypt [None req-56c942be-2ecd-493a-b874-09b6b3e4785b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81472) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 00:06:28.894421 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG passlib.handlers.bcrypt [None req-56c942be-2ecd-493a-b874-09b6b3e4785b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81472) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 00:06:28.952862 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-56c942be-2ecd-493a-b874-09b6b3e4785b None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:28.954592 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-56c942be-2ecd-493a-b874-09b6b3e4785b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:29.083132 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 2/3] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 31 00:06:29.597934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef20403-87ee-4334-9a80-413ed30f31ae None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:29.667869 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: INFO dbcounter [None req-9ef20403-87ee-4334-9a80-413ed30f31ae None None] Registered counter for database keystone Aug 31 00:06:29.689441 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG oslo_db.sqlalchemy.engines [None req-9ef20403-87ee-4334-9a80-413ed30f31ae None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81471) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 00:06:29.695650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writer thread running {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 00:06:29.814649 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef20403-87ee-4334-9a80-413ed30f31ae None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:29.818870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b2251-4dad-49dc-89a3-8c01b403c2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:29.819513 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b2251-4dad-49dc-89a3-8c01b403c2aa admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:29.819790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b2251-4dad-49dc-89a3-8c01b403c2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:29.820058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b2251-4dad-49dc-89a3-8c01b403c2aa admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:29.829074 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b2251-4dad-49dc-89a3-8c01b403c2aa admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:29.884195 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: WARNING keystone.common.rbac_enforcer.enforcer [None req-094b2251-4dad-49dc-89a3-8c01b403c2aa admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 00:06:29.884466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b2251-4dad-49dc-89a3-8c01b403c2aa admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:29.894404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 2/4] 10.222.0.67 () {58 vars in 1324 bytes} [Wed Aug 31 00:06:29 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:06:29.900411 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c942be-2ecd-493a-b874-09b6b3e4785b None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:29.914879 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c942be-2ecd-493a-b874-09b6b3e4785b None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:29.919327 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3c2d8-4d19-477b-a4ea-f680f98cb7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:29.920419 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3c2d8-4d19-477b-a4ea-f680f98cb7e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:29.920701 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3c2d8-4d19-477b-a4ea-f680f98cb7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:29.920982 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3c2d8-4d19-477b-a4ea-f680f98cb7e3 admin admin] PATH_INFO: `/v3/projects/1509e8ef0c6042559698c40a7a07a646` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:29.923071 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce3c2d8-4d19-477b-a4ea-f680f98cb7e3 admin admin] RBAC: Authorizing `identity:get_project(project_id=1509e8ef0c6042559698c40a7a07a646)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:29.975225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6ce3c2d8-4d19-477b-a4ea-f680f98cb7e3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 00:06:29.975573 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce3c2d8-4d19-477b-a4ea-f680f98cb7e3 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:29.980154 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 3/5] 10.222.0.67 () {56 vars in 1207 bytes} [Wed Aug 31 00:06:29 2022] GET /identity/v3/projects/1509e8ef0c6042559698c40a7a07a646 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:06:31.855058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af09fb-475f-41c3-aab1-7179e5f8bd1d None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:31.855591 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af09fb-475f-41c3-aab1-7179e5f8bd1d None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:31.855698 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af09fb-475f-41c3-aab1-7179e5f8bd1d None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:31.861138 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 3/6] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:06:31.869148 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d6bd1-c962-4bae-91f3-ae131e2b6b88 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:31.869605 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d6bd1-c962-4bae-91f3-ae131e2b6b88 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:31.869700 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d6bd1-c962-4bae-91f3-ae131e2b6b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:31.889842 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-3e5d6bd1-c962-4bae-91f3-ae131e2b6b88 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:31.891406 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-3e5d6bd1-c962-4bae-91f3-ae131e2b6b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:31.899276 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 4/7] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 31 00:06:32.168205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af09fb-475f-41c3-aab1-7179e5f8bd1d None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:32.180982 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af09fb-475f-41c3-aab1-7179e5f8bd1d None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:32.185192 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44c314-46af-4dd0-9bda-5d2990ed5372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:32.185800 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44c314-46af-4dd0-9bda-5d2990ed5372 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:32.186086 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44c314-46af-4dd0-9bda-5d2990ed5372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:32.186363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44c314-46af-4dd0-9bda-5d2990ed5372 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:32.195069 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44c314-46af-4dd0-9bda-5d2990ed5372 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:32.196885 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44c314-46af-4dd0-9bda-5d2990ed5372 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:32.205964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 4/8] 10.222.0.67 () {58 vars in 1324 bytes} [Wed Aug 31 00:06:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:06:32.211487 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d6bd1-c962-4bae-91f3-ae131e2b6b88 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:32.220607 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d6bd1-c962-4bae-91f3-ae131e2b6b88 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:32.224913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7936815-4e97-4bd5-9256-6be73859e445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:32.225532 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7936815-4e97-4bd5-9256-6be73859e445 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:32.225812 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7936815-4e97-4bd5-9256-6be73859e445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:32.226084 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7936815-4e97-4bd5-9256-6be73859e445 admin admin] PATH_INFO: `/v3/users/19ddd6d02b884bcab507cb5ed1181266` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:32.227418 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7936815-4e97-4bd5-9256-6be73859e445 admin admin] RBAC: Authorizing `identity:get_user(user_id=19ddd6d02b884bcab507cb5ed1181266)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:32.229221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7936815-4e97-4bd5-9256-6be73859e445 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:32.230716 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 5/9] 10.222.0.67 () {56 vars in 1198 bytes} [Wed Aug 31 00:06:32 2022] GET /identity/v3/users/19ddd6d02b884bcab507cb5ed1181266 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:34.231059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50452b3a-6650-41c0-be3b-0a3f2737c53a None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:34.231595 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50452b3a-6650-41c0-be3b-0a3f2737c53a None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:34.231677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50452b3a-6650-41c0-be3b-0a3f2737c53a None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:34.232959 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 5/10] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:06:34.238854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62941c-b728-435d-a2fd-07e8f55c4f09 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:34.239304 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62941c-b728-435d-a2fd-07e8f55c4f09 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:34.239574 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62941c-b728-435d-a2fd-07e8f55c4f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:34.258269 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-db62941c-b728-435d-a2fd-07e8f55c4f09 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:34.259378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-db62941c-b728-435d-a2fd-07e8f55c4f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:34.264157 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106de11-e62a-4a45-bc95-3d49ec2d8e33 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:34.264447 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106de11-e62a-4a45-bc95-3d49ec2d8e33 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:34.264725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106de11-e62a-4a45-bc95-3d49ec2d8e33 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:34.266959 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 6/11] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:06:34.271999 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345c83d-9c75-4665-a565-e61165054cb5 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:34.272286 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345c83d-9c75-4665-a565-e61165054cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:34.272566 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345c83d-9c75-4665-a565-e61165054cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:34.316482 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG passlib.handlers.bcrypt [None req-2345c83d-9c75-4665-a565-e61165054cb5 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81471) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 00:06:34.316882 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG passlib.handlers.bcrypt [None req-2345c83d-9c75-4665-a565-e61165054cb5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81471) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 00:06:34.339135 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 6/12] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:06:34.366702 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-2345c83d-9c75-4665-a565-e61165054cb5 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:34.368342 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-2345c83d-9c75-4665-a565-e61165054cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:34.471336 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 7/13] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:06:34.607221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62941c-b728-435d-a2fd-07e8f55c4f09 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:34.632411 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:34.715765 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62941c-b728-435d-a2fd-07e8f55c4f09 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:34.747354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72bdcf-bfb9-4fc6-b1fb-5029c56f5c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:34.748051 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72bdcf-bfb9-4fc6-b1fb-5029c56f5c40 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:34.748338 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72bdcf-bfb9-4fc6-b1fb-5029c56f5c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:34.748611 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72bdcf-bfb9-4fc6-b1fb-5029c56f5c40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:34.748700 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345c83d-9c75-4665-a565-e61165054cb5 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:34.761746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f72bdcf-bfb9-4fc6-b1fb-5029c56f5c40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:06:34.770300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-6f72bdcf-bfb9-4fc6-b1fb-5029c56f5c40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:06:34.770300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:06:34.770300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:06:34.770300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:06:34.770300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:06:34.770300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:06:34.770300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:06:34.770300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:06:34.770300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:06:34.770300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:06:34.770300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:06:34.770300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:06:34.770300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:06:34.770300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:06:34.770300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:06:34.770300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:06:34.771491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:06:34.771491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:06:34.771491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:06:34.771491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:06:34.771491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:06:34.771491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:06:34.771491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:06:34.771491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:06:34.771491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:06:34.771491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:06:34.771491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:06:34.771491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:06:34.771491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:06:34.771491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:06:34.771491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:06:34.771491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:06:34.772370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:06:34.772370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:06:34.772370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:06:34.772370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:06:34.772370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:06:34.772370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:06:34.772370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:06:34.772370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:06:34.772370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:06:34.772370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:06:34.772370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:06:34.772370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:06:34.772370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 7/14] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:06:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:06:34.777466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72bdcf-bfb9-4fc6-b1fb-5029c56f5c40 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:34.853344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345c83d-9c75-4665-a565-e61165054cb5 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:34.855708 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:34.859899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:34.878982 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72bdcf-bfb9-4fc6-b1fb-5029c56f5c40 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:34.892462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d420c-698a-46c7-8921-932360c6ee86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:34.893336 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d420c-698a-46c7-8921-932360c6ee86 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:34.893619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d420c-698a-46c7-8921-932360c6ee86 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:34.893892 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d420c-698a-46c7-8921-932360c6ee86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:34.912872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244ec12-1b94-47f9-aebd-7ad452d8c075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:34.913492 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244ec12-1b94-47f9-aebd-7ad452d8c075 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:34.913775 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244ec12-1b94-47f9-aebd-7ad452d8c075 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:34.914048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244ec12-1b94-47f9-aebd-7ad452d8c075 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:34.914717 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7244ec12-1b94-47f9-aebd-7ad452d8c075 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:06:34.915023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7244ec12-1b94-47f9-aebd-7ad452d8c075 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:34.916487 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7244ec12-1b94-47f9-aebd-7ad452d8c075 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:34.928640 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 8/15] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:06:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:06:34.934260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244ec12-1b94-47f9-aebd-7ad452d8c075 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:34.963481 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d420c-698a-46c7-8921-932360c6ee86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:34.965007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d420c-698a-46c7-8921-932360c6ee86 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:35.032930 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244ec12-1b94-47f9-aebd-7ad452d8c075 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:35.066219 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 8/16] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:06:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 00:06:35.067962 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a57922-8f36-466d-9862-056683b2fbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:35.068600 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a57922-8f36-466d-9862-056683b2fbaa None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:35.068885 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a57922-8f36-466d-9862-056683b2fbaa None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:35.069155 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a57922-8f36-466d-9862-056683b2fbaa None admin] PATH_INFO: `/v3/users/19ddd6d02b884bcab507cb5ed1181266` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:35.070314 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a57922-8f36-466d-9862-056683b2fbaa None admin] RBAC: Authorizing `identity:get_user(user_id=19ddd6d02b884bcab507cb5ed1181266)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:35.071942 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d420c-698a-46c7-8921-932360c6ee86 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:35.072072 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a57922-8f36-466d-9862-056683b2fbaa None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:35.073635 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 9/17] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:06:34 2022] GET /identity/v3/users/19ddd6d02b884bcab507cb5ed1181266 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:06:35.078951 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a57922-8f36-466d-9862-056683b2fbaa None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:35.079573 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:35.092940 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:35.175732 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d420c-698a-46c7-8921-932360c6ee86 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:35.178379 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a57922-8f36-466d-9862-056683b2fbaa None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:35.208982 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c77257-e461-4c2d-8c86-6740e5886450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:35.209831 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c77257-e461-4c2d-8c86-6740e5886450 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:35.210114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c77257-e461-4c2d-8c86-6740e5886450 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:35.210386 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c77257-e461-4c2d-8c86-6740e5886450 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:35.212599 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45837d5-d818-4cf5-95b0-bf17aa13d7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:35.213293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45837d5-d818-4cf5-95b0-bf17aa13d7be None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:35.213637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45837d5-d818-4cf5-95b0-bf17aa13d7be None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:35.213973 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45837d5-d818-4cf5-95b0-bf17aa13d7be None admin] PATH_INFO: `/v3/domains/default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:35.215779 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45837d5-d818-4cf5-95b0-bf17aa13d7be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:35.217220 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45837d5-d818-4cf5-95b0-bf17aa13d7be None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:35.218945 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 10/18] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:06:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:06:35.224815 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45837d5-d818-4cf5-95b0-bf17aa13d7be None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:35.228834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c77257-e461-4c2d-8c86-6740e5886450 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:35.230305 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c77257-e461-4c2d-8c86-6740e5886450 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:35.249386 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-b0c77257-e461-4c2d-8c86-6740e5886450 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:06:35.249386 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:06:35.249386 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:06:35.249386 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:06:35.249386 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:06:35.249386 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:06:35.249386 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:06:35.249386 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:06:35.249386 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:06:35.249386 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:06:35.249386 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:06:35.249386 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:06:35.249386 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:06:35.249386 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:06:35.249386 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:06:35.249386 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:06:35.251294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:06:35.251294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:06:35.251294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:06:35.251294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:06:35.251294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:06:35.251294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:06:35.251294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:06:35.251294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:06:35.251294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:06:35.251294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:06:35.251294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:06:35.251294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:06:35.251294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:06:35.251294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:06:35.251294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:06:35.251294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:06:35.251294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:06:35.252356 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:06:35.252356 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:06:35.252356 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:06:35.252356 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 9/19] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:06:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:06:35.257152 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c77257-e461-4c2d-8c86-6740e5886450 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:35.307370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:35.315901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:35.327450 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45837d5-d818-4cf5-95b0-bf17aa13d7be None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:35.362039 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c77257-e461-4c2d-8c86-6740e5886450 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:35.362559 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529581b6-c99b-4c02-81fd-59ba7fa7ccd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:35.363214 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529581b6-c99b-4c02-81fd-59ba7fa7ccd5 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:35.363503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529581b6-c99b-4c02-81fd-59ba7fa7ccd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:35.363778 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529581b6-c99b-4c02-81fd-59ba7fa7ccd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:35.364513 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529581b6-c99b-4c02-81fd-59ba7fa7ccd5 None admin] RBAC: Adding query filter params (role.id=34df06ce84ad4cf8b3bd1a4490228017, scope.domain.id=default, user.id=19ddd6d02b884bcab507cb5ed1181266) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:06:35.364822 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529581b6-c99b-4c02-81fd-59ba7fa7ccd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:35.366284 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529581b6-c99b-4c02-81fd-59ba7fa7ccd5 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:35.372796 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 11/20] 10.222.0.67 () {56 vars in 1322 bytes} [Wed Aug 31 00:06:35 2022] GET /identity/v3/role_assignments?user.id=19ddd6d02b884bcab507cb5ed1181266&scope.domain.id=default&role.id=34df06ce84ad4cf8b3bd1a4490228017 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:06:35.396549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb8ec3-5b59-404a-a1f7-4cab3f88687c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:35.397192 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb8ec3-5b59-404a-a1f7-4cab3f88687c None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:35.397472 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb8ec3-5b59-404a-a1f7-4cab3f88687c None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:35.397747 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb8ec3-5b59-404a-a1f7-4cab3f88687c None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:35.398411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bb8ec3-5b59-404a-a1f7-4cab3f88687c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:06:35.398715 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bb8ec3-5b59-404a-a1f7-4cab3f88687c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:35.400185 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bb8ec3-5b59-404a-a1f7-4cab3f88687c None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:35.420000 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 10/21] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:06:37.663209 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c9ab6-5f7b-40c3-9842-e90de27ff1c3 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:37.664243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c9ab6-5f7b-40c3-9842-e90de27ff1c3 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:37.664729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c9ab6-5f7b-40c3-9842-e90de27ff1c3 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:37.666275 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 12/22] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:06:37.671991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26278b-d1a8-41a6-b5a7-e6bb4783086e None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:37.672894 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26278b-d1a8-41a6-b5a7-e6bb4783086e None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:37.673369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26278b-d1a8-41a6-b5a7-e6bb4783086e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:37.693448 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-6f26278b-d1a8-41a6-b5a7-e6bb4783086e None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:37.695170 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-6f26278b-d1a8-41a6-b5a7-e6bb4783086e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:37.799755 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 11/23] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:06:37.962816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482073b0-4834-4fb0-b5ba-93b7420f1144 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:37.963454 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482073b0-4834-4fb0-b5ba-93b7420f1144 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:37.964100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482073b0-4834-4fb0-b5ba-93b7420f1144 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:37.965737 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 13/24] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:06:37.971757 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658b7eb-2de9-48e2-b19a-61af5e27648e None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:37.972366 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658b7eb-2de9-48e2-b19a-61af5e27648e None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:37.972838 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658b7eb-2de9-48e2-b19a-61af5e27648e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:37.999907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-0658b7eb-2de9-48e2-b19a-61af5e27648e None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:38.002376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e0ac8-47fc-4152-ad88-a8a7dea7adff None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:38.002376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e0ac8-47fc-4152-ad88-a8a7dea7adff None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:38.002376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e0ac8-47fc-4152-ad88-a8a7dea7adff None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:38.002666 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-0658b7eb-2de9-48e2-b19a-61af5e27648e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:38.002747 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 14/25] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:06:38.009027 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e1095-63f8-4dba-a2c2-de2d89b5007a None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:38.009588 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e1095-63f8-4dba-a2c2-de2d89b5007a None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:38.010090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e1095-63f8-4dba-a2c2-de2d89b5007a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:38.060022 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-ae0e1095-63f8-4dba-a2c2-de2d89b5007a None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:38.061251 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-ae0e1095-63f8-4dba-a2c2-de2d89b5007a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:38.091125 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:38.146837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 12/26] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:06:38.148242 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9bee5-8fcb-40fe-a843-ca974649f497 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:38.148584 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9bee5-8fcb-40fe-a843-ca974649f497 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:38.148872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9bee5-8fcb-40fe-a843-ca974649f497 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:38.151781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 13/27] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:06:38.160503 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143357c-098b-4eb3-bd62-1860d43d2752 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:38.161752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143357c-098b-4eb3-bd62-1860d43d2752 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:38.162853 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143357c-098b-4eb3-bd62-1860d43d2752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:38.196718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-5143357c-098b-4eb3-bd62-1860d43d2752 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:38.201184 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-5143357c-098b-4eb3-bd62-1860d43d2752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:38.206215 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 15/28] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:06:38.294592 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 14/29] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:06:38.333784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e1095-63f8-4dba-a2c2-de2d89b5007a None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:38.423722 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:38.460324 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e1095-63f8-4dba-a2c2-de2d89b5007a None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:38.512631 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffec54-8a6b-4bb1-b181-3f76121212a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:38.513279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffec54-8a6b-4bb1-b181-3f76121212a7 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:38.513562 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffec54-8a6b-4bb1-b181-3f76121212a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:38.513836 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffec54-8a6b-4bb1-b181-3f76121212a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:38.525296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffec54-8a6b-4bb1-b181-3f76121212a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:06:38.530383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-cfffec54-8a6b-4bb1-b181-3f76121212a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:06:38.530383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:06:38.530383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:06:38.530383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:06:38.530383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:06:38.530383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:06:38.530383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:06:38.530383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:06:38.530383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:06:38.530383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:06:38.530383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:06:38.530383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:06:38.530383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:06:38.530383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:06:38.530383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:06:38.530383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:06:38.531503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:06:38.531503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:06:38.531503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:06:38.531503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:06:38.531503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:06:38.531503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:06:38.531503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:06:38.531503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:06:38.531503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:06:38.531503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:06:38.531503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:06:38.531503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:06:38.531503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:06:38.531503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:06:38.531503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:06:38.531503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:06:38.532570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:06:38.532570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:06:38.532570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:06:38.532570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:06:38.532570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:06:38.532570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:06:38.532570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:06:38.532570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:06:38.532570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:06:38.532570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:06:38.532570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:06:38.532570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:06:38.532570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 16/30] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:06:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:06:38.537770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5143357c-098b-4eb3-bd62-1860d43d2752 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:38.549518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffec54-8a6b-4bb1-b181-3f76121212a7 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:38.700584 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5143357c-098b-4eb3-bd62-1860d43d2752 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:38.713509 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:38.758512 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffec54-8a6b-4bb1-b181-3f76121212a7 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:38.763753 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd05e6-055d-4e86-935f-26b9fe057d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:38.764392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd05e6-055d-4e86-935f-26b9fe057d44 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:38.764676 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd05e6-055d-4e86-935f-26b9fe057d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:38.764992 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd05e6-055d-4e86-935f-26b9fe057d44 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:38.765681 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd05e6-055d-4e86-935f-26b9fe057d44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:06:38.765977 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd05e6-055d-4e86-935f-26b9fe057d44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:38.767393 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd05e6-055d-4e86-935f-26b9fe057d44 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:38.780042 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 15/31] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:06:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:38.780771 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd05e6-055d-4e86-935f-26b9fe057d44 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:38.835857 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c958bc-b25e-472c-bafe-b3063343f773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:38.837279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c958bc-b25e-472c-bafe-b3063343f773 None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:38.837564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c958bc-b25e-472c-bafe-b3063343f773 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:38.837838 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c958bc-b25e-472c-bafe-b3063343f773 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:38.838530 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c958bc-b25e-472c-bafe-b3063343f773 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:38.839953 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c958bc-b25e-472c-bafe-b3063343f773 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:38.844889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:38.918547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd05e6-055d-4e86-935f-26b9fe057d44 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:38.955860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd354b2f-902a-46c0-82cc-e980ad69d02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:38.956848 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd354b2f-902a-46c0-82cc-e980ad69d02d None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:38.957335 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd354b2f-902a-46c0-82cc-e980ad69d02d None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:38.957808 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd354b2f-902a-46c0-82cc-e980ad69d02d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:38.975951 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd354b2f-902a-46c0-82cc-e980ad69d02d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:38.977441 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd354b2f-902a-46c0-82cc-e980ad69d02d None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:38.995575 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 17/32] 10.222.0.67 () {60 vars in 1131 bytes} [Wed Aug 31 00:06:38 2022] POST /identity/v3/roles => generated 225 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:06:38.996946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c958bc-b25e-472c-bafe-b3063343f773 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:38.999114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-fd354b2f-902a-46c0-82cc-e980ad69d02d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:06:38.999114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:06:38.999114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:06:38.999114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:06:38.999114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:06:38.999114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:06:38.999114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:06:38.999114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:06:38.999114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:06:38.999114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:06:38.999114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:06:38.999114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:06:38.999114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:06:38.999114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:06:38.999114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:06:38.999114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:06:39.000054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:06:39.000054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:06:39.000054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:06:39.000054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:06:39.000054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:06:39.000054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:06:39.000054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:06:39.000054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:06:39.000054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:06:39.000054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:06:39.000054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:06:39.000054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:06:39.000054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:06:39.000054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:06:39.000054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:06:39.000054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:06:39.000054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:06:39.000865 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:06:39.000865 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:06:39.000865 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:06:39.000865 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 16/33] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:06:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:06:39.006950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd354b2f-902a-46c0-82cc-e980ad69d02d None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:39.059061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:39.131997 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c958bc-b25e-472c-bafe-b3063343f773 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:39.173595 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68479c-fb2a-48a2-b777-22a163ef0659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:39.174255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68479c-fb2a-48a2-b777-22a163ef0659 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:39.174537 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68479c-fb2a-48a2-b777-22a163ef0659 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:39.174816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68479c-fb2a-48a2-b777-22a163ef0659 None admin] PATH_INFO: `/v3/users/19ddd6d02b884bcab507cb5ed1181266` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:39.176832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68479c-fb2a-48a2-b777-22a163ef0659 None admin] RBAC: Authorizing `identity:get_user(user_id=19ddd6d02b884bcab507cb5ed1181266)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:39.180762 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68479c-fb2a-48a2-b777-22a163ef0659 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:39.180762 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 18/34] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:06:38 2022] GET /identity/v3/users/19ddd6d02b884bcab507cb5ed1181266 => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:39.181292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68479c-fb2a-48a2-b777-22a163ef0659 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:39.241465 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd354b2f-902a-46c0-82cc-e980ad69d02d None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:39.259287 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:39.287938 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c705a7c9-1bdb-46d8-ba4d-72ef5372aaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:39.288865 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705a7c9-1bdb-46d8-ba4d-72ef5372aaa8 None admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:39.289169 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705a7c9-1bdb-46d8-ba4d-72ef5372aaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:39.289443 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705a7c9-1bdb-46d8-ba4d-72ef5372aaa8 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:39.290122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c705a7c9-1bdb-46d8-ba4d-72ef5372aaa8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:39.291550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c705a7c9-1bdb-46d8-ba4d-72ef5372aaa8 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:39.318945 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68479c-fb2a-48a2-b777-22a163ef0659 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:39.360179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1962829-7112-430b-80cc-3d22978fc745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:39.361192 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1962829-7112-430b-80cc-3d22978fc745 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:39.361694 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1962829-7112-430b-80cc-3d22978fc745 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:39.362183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1962829-7112-430b-80cc-3d22978fc745 None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:39.363123 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1962829-7112-430b-80cc-3d22978fc745 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:06:39.363683 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1962829-7112-430b-80cc-3d22978fc745 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:39.365294 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1962829-7112-430b-80cc-3d22978fc745 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:39.385476 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 19/35] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:39.386582 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1962829-7112-430b-80cc-3d22978fc745 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:39.431094 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 17/36] 10.222.0.67 () {60 vars in 1131 bytes} [Wed Aug 31 00:06:39 2022] POST /identity/v3/roles => generated 219 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:06:39.432762 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862266f7-d43c-4fd0-a058-92c9524bfa86 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:39.433061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862266f7-d43c-4fd0-a058-92c9524bfa86 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:39.433345 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862266f7-d43c-4fd0-a058-92c9524bfa86 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:39.434922 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 18/37] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:06:39.435840 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862266f7-d43c-4fd0-a058-92c9524bfa86 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:39.503751 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1962829-7112-430b-80cc-3d22978fc745 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:39.530331 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:39.537503 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:39.544539 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6561e-af15-4f78-a0d4-42abc9ddb261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:39.545593 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6561e-af15-4f78-a0d4-42abc9ddb261 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:39.545905 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6561e-af15-4f78-a0d4-42abc9ddb261 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:39.546708 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6561e-af15-4f78-a0d4-42abc9ddb261 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:39.548170 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6561e-af15-4f78-a0d4-42abc9ddb261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:39.549750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6561e-af15-4f78-a0d4-42abc9ddb261 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:39.551957 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 20/38] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:06:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:39.553034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19921ca2-7d57-469b-b94a-b749aeac0681 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:39.553322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19921ca2-7d57-469b-b94a-b749aeac0681 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:39.553598 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19921ca2-7d57-469b-b94a-b749aeac0681 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:39.555106 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 21/39] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:06:39.560190 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91117b-08e6-4d0b-9091-8b5ffd4db7bc None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:39.560483 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91117b-08e6-4d0b-9091-8b5ffd4db7bc None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:39.560769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91117b-08e6-4d0b-9091-8b5ffd4db7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:39.570239 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862266f7-d43c-4fd0-a058-92c9524bfa86 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:39.586093 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-cd91117b-08e6-4d0b-9091-8b5ffd4db7bc None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:39.587649 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-cd91117b-08e6-4d0b-9091-8b5ffd4db7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:39.621544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d24d1-3a97-4c70-856c-581bb964221e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:39.622175 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d24d1-3a97-4c70-856c-581bb964221e None admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:39.622462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d24d1-3a97-4c70-856c-581bb964221e None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:39.622742 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d24d1-3a97-4c70-856c-581bb964221e None admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:39.623723 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493d24d1-3a97-4c70-856c-581bb964221e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:39.627250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493d24d1-3a97-4c70-856c-581bb964221e None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:39.660399 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 19/40] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 31 00:06:39 2022] POST /identity/v3/projects => generated 301 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:06:39.661238 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d24d1-3a97-4c70-856c-581bb964221e None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:39.678143 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 22/41] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:06:39.678143 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09d7b7-eaa9-468b-b8eb-80a13abec942 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:39.678143 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09d7b7-eaa9-468b-b8eb-80a13abec942 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:39.678143 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09d7b7-eaa9-468b-b8eb-80a13abec942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:39.694078 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-9c09d7b7-eaa9-468b-b8eb-80a13abec942 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:39.695590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-9c09d7b7-eaa9-468b-b8eb-80a13abec942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:39.794176 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 23/42] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:06:39.810858 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d24d1-3a97-4c70-856c-581bb964221e None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:39.843813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:39.852227 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887423cf-cc4c-407e-9b92-9b579218a24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:39.852931 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887423cf-cc4c-407e-9b92-9b579218a24f None admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:39.853202 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887423cf-cc4c-407e-9b92-9b579218a24f None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:39.853522 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887423cf-cc4c-407e-9b92-9b579218a24f None admin] PATH_INFO: `/v3/domains/default/users/19ddd6d02b884bcab507cb5ed1181266/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:39.856255 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887423cf-cc4c-407e-9b92-9b579218a24f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=19ddd6d02b884bcab507cb5ed1181266, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:39.857779 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887423cf-cc4c-407e-9b92-9b579218a24f None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:39.879732 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 20/43] 10.222.0.67 () {58 vars in 1361 bytes} [Wed Aug 31 00:06:39 2022] PUT /identity/v3/domains/default/users/19ddd6d02b884bcab507cb5ed1181266/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:06:39.964053 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09d7b7-eaa9-468b-b8eb-80a13abec942 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:40.026417 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:40.084979 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09d7b7-eaa9-468b-b8eb-80a13abec942 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:40.086805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887423cf-cc4c-407e-9b92-9b579218a24f None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:40.123237 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c415f-e368-490a-a2af-50ca597e73d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:40.124024 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c415f-e368-490a-a2af-50ca597e73d2 None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:40.124337 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c415f-e368-490a-a2af-50ca597e73d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:40.124616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c415f-e368-490a-a2af-50ca597e73d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:40.125595 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c415f-e368-490a-a2af-50ca597e73d2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:40.127521 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c415f-e368-490a-a2af-50ca597e73d2 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:40.143424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 24/44] 10.222.0.67 () {60 vars in 1131 bytes} [Wed Aug 31 00:06:39 2022] POST /identity/v3/roles => generated 223 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:06:40.202017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887423cf-cc4c-407e-9b92-9b579218a24f None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:40.240246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebef07c-118d-4f5e-8405-3a0574b9e8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:40.240903 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebef07c-118d-4f5e-8405-3a0574b9e8ef None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:40.241193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebef07c-118d-4f5e-8405-3a0574b9e8ef None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:40.241471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebef07c-118d-4f5e-8405-3a0574b9e8ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:40.242669 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebef07c-118d-4f5e-8405-3a0574b9e8ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:40.244185 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebef07c-118d-4f5e-8405-3a0574b9e8ef None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:40.245988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 21/45] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:06:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:06:40.252175 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c415f-e368-490a-a2af-50ca597e73d2 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:40.476739 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c415f-e368-490a-a2af-50ca597e73d2 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:40.487929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:40.513427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69233a-ba4f-4d1f-89a5-d0614ed23ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:40.513976 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69233a-ba4f-4d1f-89a5-d0614ed23ab6 None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:40.514256 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69233a-ba4f-4d1f-89a5-d0614ed23ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:40.514530 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69233a-ba4f-4d1f-89a5-d0614ed23ab6 None admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:40.515252 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e69233a-ba4f-4d1f-89a5-d0614ed23ab6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:40.516800 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e69233a-ba4f-4d1f-89a5-d0614ed23ab6 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:40.544410 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 25/46] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 31 00:06:40 2022] POST /identity/v3/projects => generated 312 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:06:41.840368 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6aa35-9bd9-453c-9207-2ad96e9e105b None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:41.840918 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6aa35-9bd9-453c-9207-2ad96e9e105b None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:41.840918 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6aa35-9bd9-453c-9207-2ad96e9e105b None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:41.842278 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 22/47] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:06:41.847726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f592c5-2969-4dbf-85b9-b29b77e32491 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:41.848130 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f592c5-2969-4dbf-85b9-b29b77e32491 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:41.848286 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f592c5-2969-4dbf-85b9-b29b77e32491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:41.866761 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-75f592c5-2969-4dbf-85b9-b29b77e32491 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:41.867955 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-75f592c5-2969-4dbf-85b9-b29b77e32491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:41.937183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 26/48] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:06:42.205048 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6aa35-9bd9-453c-9207-2ad96e9e105b None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:42.266765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:42.309182 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6aa35-9bd9-453c-9207-2ad96e9e105b None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:42.342857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bcf51-fe20-4f21-b55b-d2d341f6d2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:42.343520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bcf51-fe20-4f21-b55b-d2d341f6d2d2 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:42.343806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bcf51-fe20-4f21-b55b-d2d341f6d2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:42.344083 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bcf51-fe20-4f21-b55b-d2d341f6d2d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:42.355748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426bcf51-fe20-4f21-b55b-d2d341f6d2d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:06:42.361901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-426bcf51-fe20-4f21-b55b-d2d341f6d2d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:06:42.361901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:06:42.361901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:06:42.361901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:06:42.361901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:06:42.361901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:06:42.361901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:06:42.361901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:06:42.361901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:06:42.361901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:06:42.361901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:06:42.361901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:06:42.361901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:06:42.361901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:06:42.361901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:06:42.361901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:06:42.362973 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:06:42.362973 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:06:42.362973 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:06:42.362973 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:06:42.362973 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:06:42.362973 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:06:42.362973 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:06:42.362973 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:06:42.362973 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:06:42.362973 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:06:42.362973 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:06:42.362973 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:06:42.362973 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:06:42.362973 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:06:42.362973 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:06:42.362973 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:06:42.363784 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:06:42.363784 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:06:42.363784 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:06:42.363784 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:06:42.363784 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:06:42.363784 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:06:42.363784 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:06:42.363784 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:06:42.363784 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:06:42.363784 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:06:42.363784 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:06:42.363784 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:06:42.363784 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 23/49] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:06:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:06:42.369103 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f592c5-2969-4dbf-85b9-b29b77e32491 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:42.460108 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:42.469763 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f592c5-2969-4dbf-85b9-b29b77e32491 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:42.475978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd5264-466a-492c-a9bc-4c86cc0e13ce None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:42.476265 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd5264-466a-492c-a9bc-4c86cc0e13ce None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:42.476550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd5264-466a-492c-a9bc-4c86cc0e13ce None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:42.478314 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 24/50] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:06:42.483233 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f387b-0f53-4890-9564-9c7e24bb4a2c None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:42.484048 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f387b-0f53-4890-9564-9c7e24bb4a2c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:42.484318 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f387b-0f53-4890-9564-9c7e24bb4a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:42.503130 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-713f387b-0f53-4890-9564-9c7e24bb4a2c None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:42.504667 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-713f387b-0f53-4890-9564-9c7e24bb4a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:42.505014 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c804a7-bb30-416f-9270-f5db637fc19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:42.505627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c804a7-bb30-416f-9270-f5db637fc19e None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:42.505905 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c804a7-bb30-416f-9270-f5db637fc19e None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:42.506175 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c804a7-bb30-416f-9270-f5db637fc19e None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:42.506833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c804a7-bb30-416f-9270-f5db637fc19e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:06:42.507168 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c804a7-bb30-416f-9270-f5db637fc19e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:42.508540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c804a7-bb30-416f-9270-f5db637fc19e None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:42.516686 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 27/51] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:06:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:42.522115 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c804a7-bb30-416f-9270-f5db637fc19e None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:42.572545 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 25/52] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:06:42.619095 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c804a7-bb30-416f-9270-f5db637fc19e None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:42.652122 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe9dd6-422c-46f9-bc5b-76abd8ce725b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:42.652763 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe9dd6-422c-46f9-bc5b-76abd8ce725b None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:42.653049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe9dd6-422c-46f9-bc5b-76abd8ce725b None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:42.653325 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe9dd6-422c-46f9-bc5b-76abd8ce725b None admin] PATH_INFO: `/v3/users/19ddd6d02b884bcab507cb5ed1181266` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:42.654502 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe9dd6-422c-46f9-bc5b-76abd8ce725b None admin] RBAC: Authorizing `identity:get_user(user_id=19ddd6d02b884bcab507cb5ed1181266)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:42.656022 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe9dd6-422c-46f9-bc5b-76abd8ce725b None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:42.657600 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 28/53] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:06:42 2022] GET /identity/v3/users/19ddd6d02b884bcab507cb5ed1181266 => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:06:42.663275 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f387b-0f53-4890-9564-9c7e24bb4a2c None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:42.713082 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:42.761601 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713f387b-0f53-4890-9564-9c7e24bb4a2c None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:42.796686 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a93711-d57d-4da7-8990-1410d1a37a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:42.797344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a93711-d57d-4da7-8990-1410d1a37a88 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:42.797628 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a93711-d57d-4da7-8990-1410d1a37a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:42.797907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a93711-d57d-4da7-8990-1410d1a37a88 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:42.799019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a93711-d57d-4da7-8990-1410d1a37a88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:42.800490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a93711-d57d-4da7-8990-1410d1a37a88 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:42.802042 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 26/54] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:06:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:06:42.807843 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe9dd6-422c-46f9-bc5b-76abd8ce725b None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:42.832782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:42.853152 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a93711-d57d-4da7-8990-1410d1a37a88 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:42.911895 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe9dd6-422c-46f9-bc5b-76abd8ce725b None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:42.949162 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dcc51-44d9-40b7-a0bd-cdadfd710ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:42.949800 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dcc51-44d9-40b7-a0bd-cdadfd710ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:42.950089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dcc51-44d9-40b7-a0bd-cdadfd710ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:42.950369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dcc51-44d9-40b7-a0bd-cdadfd710ab8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:42.951152 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40dcc51-44d9-40b7-a0bd-cdadfd710ab8 None admin] RBAC: Adding query filter params (role.id=34df06ce84ad4cf8b3bd1a4490228017, scope.domain.id=default, user.id=19ddd6d02b884bcab507cb5ed1181266) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:06:42.951470 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40dcc51-44d9-40b7-a0bd-cdadfd710ab8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:42.953946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40dcc51-44d9-40b7-a0bd-cdadfd710ab8 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:42.960542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 29/55] 10.222.0.67 () {56 vars in 1322 bytes} [Wed Aug 31 00:06:42 2022] GET /identity/v3/role_assignments?user.id=19ddd6d02b884bcab507cb5ed1181266&scope.domain.id=default&role.id=34df06ce84ad4cf8b3bd1a4490228017 => generated 530 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:43.067130 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a93711-d57d-4da7-8990-1410d1a37a88 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:43.084988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:43.103046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d7627e-fd1a-4223-94fc-6cc36fdbdde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:43.103689 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7627e-fd1a-4223-94fc-6cc36fdbdde6 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:43.103972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7627e-fd1a-4223-94fc-6cc36fdbdde6 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:43.104243 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7627e-fd1a-4223-94fc-6cc36fdbdde6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:43.105340 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d7627e-fd1a-4223-94fc-6cc36fdbdde6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:43.106806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d7627e-fd1a-4223-94fc-6cc36fdbdde6 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:43.109105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 27/56] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:06:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:06:43.116865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dcc51-44d9-40b7-a0bd-cdadfd710ab8 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:43.208981 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:43.224412 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dcc51-44d9-40b7-a0bd-cdadfd710ab8 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:43.256216 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d454b-6783-4e03-8b56-5999b452c3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:43.256758 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d454b-6783-4e03-8b56-5999b452c3b8 None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:43.257039 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d454b-6783-4e03-8b56-5999b452c3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:43.257310 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d454b-6783-4e03-8b56-5999b452c3b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:43.257970 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d454b-6783-4e03-8b56-5999b452c3b8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:43.259445 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d454b-6783-4e03-8b56-5999b452c3b8 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:43.287129 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 30/57] 10.222.0.67 () {60 vars in 1140 bytes} [Wed Aug 31 00:06:43 2022] POST /identity/v3/projects => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:06:45.204071 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ef57e-5f6b-45f3-8a74-fa09191d2023 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:45.204574 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ef57e-5f6b-45f3-8a74-fa09191d2023 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:45.204660 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ef57e-5f6b-45f3-8a74-fa09191d2023 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:45.205974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 28/58] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:06:45.211196 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718e250-161c-48e1-a7a4-17ebfcfedab5 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:45.211584 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718e250-161c-48e1-a7a4-17ebfcfedab5 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:45.211749 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718e250-161c-48e1-a7a4-17ebfcfedab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:45.230477 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-2718e250-161c-48e1-a7a4-17ebfcfedab5 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:45.231726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-2718e250-161c-48e1-a7a4-17ebfcfedab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:45.308720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 31/59] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:06:45.589236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ef57e-5f6b-45f3-8a74-fa09191d2023 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:45.696439 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ef57e-5f6b-45f3-8a74-fa09191d2023 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:45.731689 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539bfb7-6686-47a4-8420-3dfcc4f8a0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:45.732325 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539bfb7-6686-47a4-8420-3dfcc4f8a0c3 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:45.732611 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539bfb7-6686-47a4-8420-3dfcc4f8a0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:45.732887 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539bfb7-6686-47a4-8420-3dfcc4f8a0c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:45.735079 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9539bfb7-6686-47a4-8420-3dfcc4f8a0c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:45.736616 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9539bfb7-6686-47a4-8420-3dfcc4f8a0c3 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:45.738258 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 29/60] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:06:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:45.743827 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718e250-161c-48e1-a7a4-17ebfcfedab5 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:45.818168 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:45.858155 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718e250-161c-48e1-a7a4-17ebfcfedab5 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:45.904053 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d7575-1620-4c06-9408-e43f2e3687da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:45.904928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d7575-1620-4c06-9408-e43f2e3687da None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:45.905745 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d7575-1620-4c06-9408-e43f2e3687da None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:45.906019 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d7575-1620-4c06-9408-e43f2e3687da None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:45.907226 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d7575-1620-4c06-9408-e43f2e3687da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:45.909103 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d7575-1620-4c06-9408-e43f2e3687da None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:45.945795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 32/61] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 31 00:06:45 2022] POST /identity/v3/users => generated 275 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:06:49.048258 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c10345-d728-4a63-a9fd-3bcba0b047e8 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:49.049270 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c10345-d728-4a63-a9fd-3bcba0b047e8 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:49.049774 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c10345-d728-4a63-a9fd-3bcba0b047e8 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:49.051377 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 30/62] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:06:49.057080 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33fe1cb-6aa7-4654-9caf-0be1e59e5cf7 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:49.057947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33fe1cb-6aa7-4654-9caf-0be1e59e5cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:49.058424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33fe1cb-6aa7-4654-9caf-0be1e59e5cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:49.078832 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66919442-01ba-47e4-a9bf-b9361bf06358 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:49.079458 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66919442-01ba-47e4-a9bf-b9361bf06358 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:49.079973 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66919442-01ba-47e4-a9bf-b9361bf06358 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:49.081516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 31/63] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:06:49.084619 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-b33fe1cb-6aa7-4654-9caf-0be1e59e5cf7 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:49.085802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b33fe1cb-6aa7-4654-9caf-0be1e59e5cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:49.098889 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ad267-f053-418c-bf00-eaeafd8b1e36 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:49.099397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ad267-f053-418c-bf00-eaeafd8b1e36 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:49.099817 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ad267-f053-418c-bf00-eaeafd8b1e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:49.132379 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-649ad267-f053-418c-bf00-eaeafd8b1e36 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:49.134698 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-649ad267-f053-418c-bf00-eaeafd8b1e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:49.195175 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 33/64] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:06:49.231422 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:49.246119 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 32/65] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:06:49.416067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed2149-7d37-4b0b-bf40-7bf670bfe31e None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:49.416435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed2149-7d37-4b0b-bf40-7bf670bfe31e None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:49.416727 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed2149-7d37-4b0b-bf40-7bf670bfe31e None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:49.418054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 34/66] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:06:49.424156 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f2a7a-5475-4ce4-ae5f-8094a1728475 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:49.424633 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f2a7a-5475-4ce4-ae5f-8094a1728475 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:49.425057 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f2a7a-5475-4ce4-ae5f-8094a1728475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:49.449831 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-cc1f2a7a-5475-4ce4-ae5f-8094a1728475 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:49.451267 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-cc1f2a7a-5475-4ce4-ae5f-8094a1728475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:49.508184 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed2149-7d37-4b0b-bf40-7bf670bfe31e None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:49.554384 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 33/67] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:06:49.555417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f2a7a-5475-4ce4-ae5f-8094a1728475 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:49.585521 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:49.660605 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed2149-7d37-4b0b-bf40-7bf670bfe31e None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:49.698010 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a50db-1354-47ea-979e-4ac599e433b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:49.698933 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a50db-1354-47ea-979e-4ac599e433b8 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:49.699413 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a50db-1354-47ea-979e-4ac599e433b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:49.699964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a50db-1354-47ea-979e-4ac599e433b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:49.710735 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a50db-1354-47ea-979e-4ac599e433b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:06:49.715677 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-937a50db-1354-47ea-979e-4ac599e433b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:06:49.715677 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:06:49.715677 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:06:49.715677 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:06:49.715677 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:06:49.715677 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:06:49.715677 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:06:49.715677 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:06:49.715677 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:06:49.715677 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:06:49.715677 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:06:49.715677 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:06:49.715677 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:06:49.715677 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:06:49.715677 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:06:49.715677 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:06:49.716507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:06:49.716507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:06:49.716507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:06:49.716507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:06:49.716507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:06:49.716507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:06:49.716507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:06:49.716507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:06:49.716507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:06:49.716507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:06:49.716507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:06:49.716507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:06:49.716507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:06:49.716507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:06:49.716507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:06:49.716507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:06:49.717255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:06:49.717255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:06:49.717255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:06:49.717255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:06:49.717255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:06:49.717255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:06:49.717255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:06:49.717255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:06:49.717255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:06:49.717255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:06:49.717255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:06:49.717255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:06:49.717255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 35/68] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:06:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:06:49.719912 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8e379-1c82-419f-86fb-b8970d768871 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:49.719912 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8e379-1c82-419f-86fb-b8970d768871 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:49.720058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8e379-1c82-419f-86fb-b8970d768871 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:49.721256 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 36/69] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:06:49.722284 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd696fef-8d06-4a1f-9c0c-dea53fc3483a None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:49.722639 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd696fef-8d06-4a1f-9c0c-dea53fc3483a None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:49.722928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd696fef-8d06-4a1f-9c0c-dea53fc3483a None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:49.724402 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 37/70] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:06:49.724839 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd696fef-8d06-4a1f-9c0c-dea53fc3483a None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:49.736822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f2a7a-5475-4ce4-ae5f-8094a1728475 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:49.753015 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:49.775441 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6b28b8-517e-4332-90bb-a806b9f6cba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:49.776005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b28b8-517e-4332-90bb-a806b9f6cba2 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:49.776290 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b28b8-517e-4332-90bb-a806b9f6cba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:49.776566 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b28b8-517e-4332-90bb-a806b9f6cba2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:49.787945 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6b28b8-517e-4332-90bb-a806b9f6cba2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:06:49.792778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-6a6b28b8-517e-4332-90bb-a806b9f6cba2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:06:49.792778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:06:49.792778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:06:49.792778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:06:49.792778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:06:49.792778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:06:49.792778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:06:49.792778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:06:49.792778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:06:49.792778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:06:49.792778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:06:49.792778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:06:49.792778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:06:49.792778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:06:49.792778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:06:49.792778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:06:49.793642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:06:49.793642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:06:49.793642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:06:49.793642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:06:49.793642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:06:49.793642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:06:49.793642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:06:49.793642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:06:49.793642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:06:49.793642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:06:49.793642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:06:49.793642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:06:49.793642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:06:49.793642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:06:49.793642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:06:49.793642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:06:49.794404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:06:49.794404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:06:49.794404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:06:49.794404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:06:49.794404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:06:49.794404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:06:49.794404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:06:49.794404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:06:49.794404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:06:49.794404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:06:49.794404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:06:49.794404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:06:49.794404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 34/71] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:06:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:06:49.796329 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cc72a-4bcd-4ab1-802c-0c8646aaee16 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:49.796548 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cc72a-4bcd-4ab1-802c-0c8646aaee16 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:49.796996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cc72a-4bcd-4ab1-802c-0c8646aaee16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:49.814791 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-dd8cc72a-4bcd-4ab1-802c-0c8646aaee16 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:49.816012 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-dd8cc72a-4bcd-4ab1-802c-0c8646aaee16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:49.850227 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd696fef-8d06-4a1f-9c0c-dea53fc3483a None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:49.867346 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:49.888391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 35/72] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:06:49.888848 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bd13f-969f-4342-a339-b951ea44cc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:49.890401 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bd13f-969f-4342-a339-b951ea44cc90 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:49.890401 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bd13f-969f-4342-a339-b951ea44cc90 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:49.890611 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bd13f-969f-4342-a339-b951ea44cc90 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:49.891293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60bd13f-969f-4342-a339-b951ea44cc90 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:06:49.891522 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a14e0-1240-49c8-b2c6-057c1fd80947 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:49.891616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60bd13f-969f-4342-a339-b951ea44cc90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:49.891808 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a14e0-1240-49c8-b2c6-057c1fd80947 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:49.892079 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a14e0-1240-49c8-b2c6-057c1fd80947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:49.892987 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60bd13f-969f-4342-a339-b951ea44cc90 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:49.902830 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 38/73] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:06:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:49.903745 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bd13f-969f-4342-a339-b951ea44cc90 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:49.911662 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-c54a14e0-1240-49c8-b2c6-057c1fd80947 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:49.912843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c54a14e0-1240-49c8-b2c6-057c1fd80947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:50.003070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 36/74] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:06:50.003070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a14e0-1240-49c8-b2c6-057c1fd80947 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:50.030589 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bd13f-969f-4342-a339-b951ea44cc90 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:50.050193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:50.069673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5484b2-b93d-445d-8a7f-e936ac32586f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:50.070281 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5484b2-b93d-445d-8a7f-e936ac32586f None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:50.070568 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5484b2-b93d-445d-8a7f-e936ac32586f None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:50.070843 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5484b2-b93d-445d-8a7f-e936ac32586f None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:50.071644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5484b2-b93d-445d-8a7f-e936ac32586f None admin] RBAC: Adding query filter params (name=member) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:06:50.072052 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5484b2-b93d-445d-8a7f-e936ac32586f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:50.073439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5484b2-b93d-445d-8a7f-e936ac32586f None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:50.084023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 39/75] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:06:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:50.084924 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5484b2-b93d-445d-8a7f-e936ac32586f None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:50.129515 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:50.138631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a14e0-1240-49c8-b2c6-057c1fd80947 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:50.179207 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbba412-5c4c-49fa-aa87-24f437bb8bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:50.179843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbba412-5c4c-49fa-aa87-24f437bb8bdf None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:50.180123 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbba412-5c4c-49fa-aa87-24f437bb8bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:50.180394 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbba412-5c4c-49fa-aa87-24f437bb8bdf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:50.190971 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbba412-5c4c-49fa-aa87-24f437bb8bdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:06:50.195822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-3cbba412-5c4c-49fa-aa87-24f437bb8bdf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:06:50.195822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:06:50.195822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:06:50.195822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:06:50.195822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:06:50.195822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:06:50.195822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:06:50.195822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:06:50.195822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:06:50.195822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:06:50.195822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:06:50.195822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:06:50.195822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:06:50.195822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:06:50.195822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:06:50.195822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:06:50.196859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:06:50.196859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:06:50.196859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:06:50.196859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:06:50.196859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:06:50.196859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:06:50.196859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:06:50.196859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:06:50.196859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:06:50.196859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:06:50.196859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:06:50.196859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:06:50.196859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:06:50.196859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:06:50.196859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:06:50.196859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:06:50.197659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:06:50.197659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:06:50.197659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:06:50.197659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:06:50.197659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:06:50.197659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:06:50.197659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:06:50.197659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:06:50.197659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:06:50.197659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:06:50.197659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:06:50.197659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:06:50.197659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 37/76] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 31 00:06:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:06:50.198276 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbba412-5c4c-49fa-aa87-24f437bb8bdf None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:50.199410 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5484b2-b93d-445d-8a7f-e936ac32586f None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:50.244836 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac993e39-1866-4c9a-af68-50f3703be64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:50.245893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac993e39-1866-4c9a-af68-50f3703be64c None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:50.246328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac993e39-1866-4c9a-af68-50f3703be64c None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:50.247147 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac993e39-1866-4c9a-af68-50f3703be64c None admin] PATH_INFO: `/v3/users/19ddd6d02b884bcab507cb5ed1181266` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:50.248787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac993e39-1866-4c9a-af68-50f3703be64c None admin] RBAC: Authorizing `identity:get_user(user_id=19ddd6d02b884bcab507cb5ed1181266)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:50.251197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac993e39-1866-4c9a-af68-50f3703be64c None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:50.253371 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 40/77] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:06:50 2022] GET /identity/v3/users/19ddd6d02b884bcab507cb5ed1181266 => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:50.254644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac993e39-1866-4c9a-af68-50f3703be64c None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:50.294610 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:50.301604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbba412-5c4c-49fa-aa87-24f437bb8bdf None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:50.336530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40b961-00e5-4d3f-88f8-5dfb9e7f45c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:50.337162 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40b961-00e5-4d3f-88f8-5dfb9e7f45c4 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:50.337442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40b961-00e5-4d3f-88f8-5dfb9e7f45c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:50.337715 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40b961-00e5-4d3f-88f8-5dfb9e7f45c4 None admin] PATH_INFO: `/v3/users/8eb6244b69ca487281ed852a375261ca` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:50.352672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c40b961-00e5-4d3f-88f8-5dfb9e7f45c4 None admin] RBAC: Authorizing `identity:get_user(user_id=8eb6244b69ca487281ed852a375261ca)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:50.354159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c40b961-00e5-4d3f-88f8-5dfb9e7f45c4 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:50.355812 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 38/78] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:06:50 2022] GET /identity/v3/users/8eb6244b69ca487281ed852a375261ca => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:50.356572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40b961-00e5-4d3f-88f8-5dfb9e7f45c4 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:50.365093 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac993e39-1866-4c9a-af68-50f3703be64c None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:50.372473 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:50.401416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a57b06-68c7-40fc-8e2c-9e09da7bb1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:50.402049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a57b06-68c7-40fc-8e2c-9e09da7bb1c8 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:50.402329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a57b06-68c7-40fc-8e2c-9e09da7bb1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:50.402603 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a57b06-68c7-40fc-8e2c-9e09da7bb1c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:50.403732 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a57b06-68c7-40fc-8e2c-9e09da7bb1c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:50.405234 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a57b06-68c7-40fc-8e2c-9e09da7bb1c8 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:50.406908 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 41/79] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:06:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:50.407725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a57b06-68c7-40fc-8e2c-9e09da7bb1c8 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:50.457755 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40b961-00e5-4d3f-88f8-5dfb9e7f45c4 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:50.490900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6b53e-c192-45a0-a62a-707e116f2f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:50.491534 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6b53e-c192-45a0-a62a-707e116f2f01 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:50.491816 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6b53e-c192-45a0-a62a-707e116f2f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:50.492086 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6b53e-c192-45a0-a62a-707e116f2f01 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:50.492746 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6b53e-c192-45a0-a62a-707e116f2f01 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:06:50.493048 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6b53e-c192-45a0-a62a-707e116f2f01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:50.494451 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6b53e-c192-45a0-a62a-707e116f2f01 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:50.503268 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 39/80] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:06:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:50.504070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6b53e-c192-45a0-a62a-707e116f2f01 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:50.508665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a57b06-68c7-40fc-8e2c-9e09da7bb1c8 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:50.519863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:50.542999 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10040b-8c01-4822-8b0a-8902573a2694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:50.543640 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10040b-8c01-4822-8b0a-8902573a2694 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:50.543920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10040b-8c01-4822-8b0a-8902573a2694 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:50.544192 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10040b-8c01-4822-8b0a-8902573a2694 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:50.587671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:50.606174 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6b53e-c192-45a0-a62a-707e116f2f01 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:50.613384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de10040b-8c01-4822-8b0a-8902573a2694 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:50.614873 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de10040b-8c01-4822-8b0a-8902573a2694 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:50.640346 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a19335-ed8d-4c0e-966c-9442d053f57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:50.640979 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a19335-ed8d-4c0e-966c-9442d053f57a None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:50.641265 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a19335-ed8d-4c0e-966c-9442d053f57a None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:50.641544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a19335-ed8d-4c0e-966c-9442d053f57a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:50.651651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a19335-ed8d-4c0e-966c-9442d053f57a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:06:50.658102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-f4a19335-ed8d-4c0e-966c-9442d053f57a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:06:50.658102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:06:50.658102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:06:50.658102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:06:50.658102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:06:50.658102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:06:50.658102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:06:50.658102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:06:50.658102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:06:50.658102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:06:50.658102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:06:50.658102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:06:50.658102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:06:50.658102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:06:50.658102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:06:50.658102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:06:50.659118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:06:50.659118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:06:50.659118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:06:50.659118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:06:50.659118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:06:50.659118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:06:50.659118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:06:50.659118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:06:50.659118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:06:50.659118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:06:50.659118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:06:50.659118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:06:50.659118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:06:50.659118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:06:50.659118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:06:50.659118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:06:50.660062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:06:50.660062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:06:50.660062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:06:50.660062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:06:50.660062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:06:50.660062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:06:50.660062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:06:50.660062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:06:50.660062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:06:50.660062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:06:50.660062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:06:50.660062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:06:50.660062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 40/81] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:06:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:06:50.660062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a19335-ed8d-4c0e-966c-9442d053f57a None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:50.719554 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 42/82] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:06:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:06:50.720314 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10040b-8c01-4822-8b0a-8902573a2694 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:50.737172 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:50.764721 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a19335-ed8d-4c0e-966c-9442d053f57a None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:50.797540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:50.800159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd185051-685d-4604-87cb-99e18e5ccbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:50.800781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd185051-685d-4604-87cb-99e18e5ccbf6 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:50.801067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd185051-685d-4604-87cb-99e18e5ccbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:50.801472 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd185051-685d-4604-87cb-99e18e5ccbf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:50.823781 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10040b-8c01-4822-8b0a-8902573a2694 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:50.858837 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6e492-5084-462c-9fd9-d297d3b494b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:50.859405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6e492-5084-462c-9fd9-d297d3b494b6 None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:50.859681 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6e492-5084-462c-9fd9-d297d3b494b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:50.860006 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6e492-5084-462c-9fd9-d297d3b494b6 None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:50.860660 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f6e492-5084-462c-9fd9-d297d3b494b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:50.862116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f6e492-5084-462c-9fd9-d297d3b494b6 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:50.869634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd185051-685d-4604-87cb-99e18e5ccbf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:50.871145 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd185051-685d-4604-87cb-99e18e5ccbf6 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:50.883650 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 43/83] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 31 00:06:50 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:06:50.884468 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6e492-5084-462c-9fd9-d297d3b494b6 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:50.955027 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:50.986983 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 41/84] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:06:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:06:50.992759 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd185051-685d-4604-87cb-99e18e5ccbf6 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:51.005991 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6e492-5084-462c-9fd9-d297d3b494b6 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:51.038760 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:51.046934 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ebc29-457d-4058-96a0-b0e6dfca79cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:51.047899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ebc29-457d-4058-96a0-b0e6dfca79cc None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:51.048367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ebc29-457d-4058-96a0-b0e6dfca79cc None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:51.048795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ebc29-457d-4058-96a0-b0e6dfca79cc None admin] PATH_INFO: `/v3/users/8eb6244b69ca487281ed852a375261ca` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:51.050147 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ebc29-457d-4058-96a0-b0e6dfca79cc None admin] RBAC: Authorizing `identity:get_user(user_id=8eb6244b69ca487281ed852a375261ca)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:51.051649 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ebc29-457d-4058-96a0-b0e6dfca79cc None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:51.053852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 44/85] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:06:50 2022] GET /identity/v3/users/8eb6244b69ca487281ed852a375261ca => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:51.054190 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ebc29-457d-4058-96a0-b0e6dfca79cc None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:51.110593 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd185051-685d-4604-87cb-99e18e5ccbf6 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:51.154186 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc6a3c-06b6-417e-bc46-68b798d279d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:51.155455 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc6a3c-06b6-417e-bc46-68b798d279d9 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:51.155690 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc6a3c-06b6-417e-bc46-68b798d279d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:51.155966 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc6a3c-06b6-417e-bc46-68b798d279d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:51.156644 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc6a3c-06b6-417e-bc46-68b798d279d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:06:51.156947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc6a3c-06b6-417e-bc46-68b798d279d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:51.158568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc6a3c-06b6-417e-bc46-68b798d279d9 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:51.167202 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ebc29-457d-4058-96a0-b0e6dfca79cc None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:51.168217 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 42/86] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:06:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:51.169135 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc6a3c-06b6-417e-bc46-68b798d279d9 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:51.206131 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c04ba00-2975-4192-94b6-1c5c3713efb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:51.206780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04ba00-2975-4192-94b6-1c5c3713efb2 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:51.207116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04ba00-2975-4192-94b6-1c5c3713efb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:51.207441 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04ba00-2975-4192-94b6-1c5c3713efb2 None admin] PATH_INFO: `/v3/projects/9eb64431e703423581c08ff8f8378eb3` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:51.208658 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c04ba00-2975-4192-94b6-1c5c3713efb2 None admin] RBAC: Authorizing `identity:get_project(project_id=9eb64431e703423581c08ff8f8378eb3)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:51.210123 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c04ba00-2975-4192-94b6-1c5c3713efb2 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:51.211959 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 45/87] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:06:51 2022] GET /identity/v3/projects/9eb64431e703423581c08ff8f8378eb3 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:51.216354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:51.218413 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c04ba00-2975-4192-94b6-1c5c3713efb2 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:51.295465 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:51.307387 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc6a3c-06b6-417e-bc46-68b798d279d9 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:51.335153 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c04ba00-2975-4192-94b6-1c5c3713efb2 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:51.351448 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1c65d-d17c-4789-8718-03475452e4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:51.352193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1c65d-d17c-4789-8718-03475452e4d5 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:51.352501 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1c65d-d17c-4789-8718-03475452e4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:51.353004 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1c65d-d17c-4789-8718-03475452e4d5 None admin] PATH_INFO: `/v3/projects/9eb64431e703423581c08ff8f8378eb3` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:51.354209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1c65d-d17c-4789-8718-03475452e4d5 None admin] RBAC: Authorizing `identity:get_project(project_id=9eb64431e703423581c08ff8f8378eb3)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:51.356413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1c65d-d17c-4789-8718-03475452e4d5 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:51.358838 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 43/88] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:06:51 2022] GET /identity/v3/projects/9eb64431e703423581c08ff8f8378eb3 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:51.364203 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1c65d-d17c-4789-8718-03475452e4d5 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:51.380350 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827338a7-5062-4de4-9709-31186cd73bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:51.381282 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827338a7-5062-4de4-9709-31186cd73bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:51.381749 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827338a7-5062-4de4-9709-31186cd73bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:51.382171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827338a7-5062-4de4-9709-31186cd73bd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:51.454887 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827338a7-5062-4de4-9709-31186cd73bd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:51.456647 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827338a7-5062-4de4-9709-31186cd73bd2 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:51.503842 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1c65d-d17c-4789-8718-03475452e4d5 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:51.507444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:51.534125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:51.556286 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b856e6-4321-4729-8e2b-193f946aafd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:51.557028 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b856e6-4321-4729-8e2b-193f946aafd2 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:51.557343 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b856e6-4321-4729-8e2b-193f946aafd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:51.557996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b856e6-4321-4729-8e2b-193f946aafd2 None admin] PATH_INFO: `/v3/users/8eb6244b69ca487281ed852a375261ca` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:51.559210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b856e6-4321-4729-8e2b-193f946aafd2 None admin] RBAC: Authorizing `identity:get_user(user_id=8eb6244b69ca487281ed852a375261ca)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:51.560685 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b856e6-4321-4729-8e2b-193f946aafd2 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:51.567078 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 44/89] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:06:51 2022] GET /identity/v3/users/8eb6244b69ca487281ed852a375261ca => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:51.567078 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b856e6-4321-4729-8e2b-193f946aafd2 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:51.583142 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 46/90] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:06:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:06:51.584011 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827338a7-5062-4de4-9709-31186cd73bd2 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:51.678533 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b856e6-4321-4729-8e2b-193f946aafd2 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:51.724577 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827338a7-5062-4de4-9709-31186cd73bd2 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:51.729594 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f66f914-2b3b-4666-8da9-e5dfa2383d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:51.730409 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66f914-2b3b-4666-8da9-e5dfa2383d82 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:51.730834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66f914-2b3b-4666-8da9-e5dfa2383d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:51.731280 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66f914-2b3b-4666-8da9-e5dfa2383d82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:51.732157 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f66f914-2b3b-4666-8da9-e5dfa2383d82 None admin] RBAC: Adding query filter params (role.id=34df06ce84ad4cf8b3bd1a4490228017, scope.project.id=9eb64431e703423581c08ff8f8378eb3, user.id=19ddd6d02b884bcab507cb5ed1181266) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:06:51.732627 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f66f914-2b3b-4666-8da9-e5dfa2383d82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:51.735448 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f66f914-2b3b-4666-8da9-e5dfa2383d82 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:51.746287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 45/91] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:06:51 2022] GET /identity/v3/role_assignments?user.id=19ddd6d02b884bcab507cb5ed1181266&scope.project.id=9eb64431e703423581c08ff8f8378eb3&role.id=34df06ce84ad4cf8b3bd1a4490228017 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:51.747329 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f66f914-2b3b-4666-8da9-e5dfa2383d82 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:51.790177 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb9037-007a-452f-9a80-7d2d0358dcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:51.791100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb9037-007a-452f-9a80-7d2d0358dcd0 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:51.791523 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb9037-007a-452f-9a80-7d2d0358dcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:51.791995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb9037-007a-452f-9a80-7d2d0358dcd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:51.793715 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb9037-007a-452f-9a80-7d2d0358dcd0 None admin] RBAC: Adding query filter params (role.id=cc8bd97c7b9745d1a52746650a2e49b9, scope.project.id=9eb64431e703423581c08ff8f8378eb3, user.id=8eb6244b69ca487281ed852a375261ca) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:06:51.794195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb9037-007a-452f-9a80-7d2d0358dcd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:51.796824 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb9037-007a-452f-9a80-7d2d0358dcd0 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:51.809663 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 47/92] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:06:51 2022] GET /identity/v3/role_assignments?user.id=8eb6244b69ca487281ed852a375261ca&scope.project.id=9eb64431e703423581c08ff8f8378eb3&role.id=cc8bd97c7b9745d1a52746650a2e49b9 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:51.810898 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb9037-007a-452f-9a80-7d2d0358dcd0 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:51.848289 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:51.853900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:51.933571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb9037-007a-452f-9a80-7d2d0358dcd0 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:51.983025 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d31e8a-ad32-4321-b3f7-4947c53453b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:51.984085 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d31e8a-ad32-4321-b3f7-4947c53453b1 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:51.984373 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d31e8a-ad32-4321-b3f7-4947c53453b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:51.984650 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d31e8a-ad32-4321-b3f7-4947c53453b1 None admin] PATH_INFO: `/v3/projects/9eb64431e703423581c08ff8f8378eb3` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:51.989316 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d31e8a-ad32-4321-b3f7-4947c53453b1 None admin] RBAC: Authorizing `identity:get_project(project_id=9eb64431e703423581c08ff8f8378eb3)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:51.990866 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d31e8a-ad32-4321-b3f7-4947c53453b1 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:51.992887 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 48/93] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:06:51 2022] GET /identity/v3/projects/9eb64431e703423581c08ff8f8378eb3 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:52.001932 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d31e8a-ad32-4321-b3f7-4947c53453b1 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:52.011086 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f66f914-2b3b-4666-8da9-e5dfa2383d82 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:52.069258 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0488baf-99f5-4ac6-a30d-c337382fec39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:52.070676 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0488baf-99f5-4ac6-a30d-c337382fec39 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:52.071210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0488baf-99f5-4ac6-a30d-c337382fec39 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:52.071689 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0488baf-99f5-4ac6-a30d-c337382fec39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:52.156848 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0488baf-99f5-4ac6-a30d-c337382fec39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:52.162259 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0488baf-99f5-4ac6-a30d-c337382fec39 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:52.213976 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:52.220893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d31e8a-ad32-4321-b3f7-4947c53453b1 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:52.239801 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:52.273141 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f18a07-9794-470b-a3a0-60b5ae10428c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:52.274907 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f18a07-9794-470b-a3a0-60b5ae10428c None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:52.275448 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f18a07-9794-470b-a3a0-60b5ae10428c None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:52.276156 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f18a07-9794-470b-a3a0-60b5ae10428c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:52.278112 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f18a07-9794-470b-a3a0-60b5ae10428c None admin] RBAC: Adding query filter params (role.id=4d91c00f40304b9e9ef4e6c89f46e5c5, scope.project.id=9eb64431e703423581c08ff8f8378eb3, user.id=8eb6244b69ca487281ed852a375261ca) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:06:52.279493 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f18a07-9794-470b-a3a0-60b5ae10428c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:52.281591 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f18a07-9794-470b-a3a0-60b5ae10428c None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:52.294115 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 49/94] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:06:52 2022] GET /identity/v3/role_assignments?user.id=8eb6244b69ca487281ed852a375261ca&scope.project.id=9eb64431e703423581c08ff8f8378eb3&role.id=4d91c00f40304b9e9ef4e6c89f46e5c5 => generated 259 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:06:52.343393 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 46/95] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:06:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:06:52.349252 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f18a07-9794-470b-a3a0-60b5ae10428c None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:52.475312 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f18a07-9794-470b-a3a0-60b5ae10428c None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:52.512897 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac9f1b-2791-459a-ba6f-1fcf907034be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:52.513928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac9f1b-2791-459a-ba6f-1fcf907034be None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:52.515095 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac9f1b-2791-459a-ba6f-1fcf907034be None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:52.515546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac9f1b-2791-459a-ba6f-1fcf907034be None admin] PATH_INFO: `/v3/projects/be6cea0351cb401184e319eef387800a` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:52.516892 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac9f1b-2791-459a-ba6f-1fcf907034be None admin] RBAC: Authorizing `identity:get_project(project_id=be6cea0351cb401184e319eef387800a)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:52.518608 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac9f1b-2791-459a-ba6f-1fcf907034be None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:52.520528 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 50/96] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:06:52 2022] GET /identity/v3/projects/be6cea0351cb401184e319eef387800a => generated 312 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:52.526935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0488baf-99f5-4ac6-a30d-c337382fec39 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:52.648722 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0488baf-99f5-4ac6-a30d-c337382fec39 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:52.696944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7920a3-33ad-491f-b5ea-db0d76a64942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:52.697793 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7920a3-33ad-491f-b5ea-db0d76a64942 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:52.698219 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7920a3-33ad-491f-b5ea-db0d76a64942 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:52.698636 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7920a3-33ad-491f-b5ea-db0d76a64942 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:52.699532 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7920a3-33ad-491f-b5ea-db0d76a64942 None admin] RBAC: Adding query filter params (role.id=cc8bd97c7b9745d1a52746650a2e49b9, scope.project.id=be6cea0351cb401184e319eef387800a, user.id=8eb6244b69ca487281ed852a375261ca) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:06:52.700006 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7920a3-33ad-491f-b5ea-db0d76a64942 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:52.702203 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7920a3-33ad-491f-b5ea-db0d76a64942 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:52.708386 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:52.712596 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 47/97] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:06:52 2022] GET /identity/v3/role_assignments?user.id=8eb6244b69ca487281ed852a375261ca&scope.project.id=be6cea0351cb401184e319eef387800a&role.id=cc8bd97c7b9745d1a52746650a2e49b9 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:53.511792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4caab-fe57-4e7e-89a6-39880956fd22 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:53.512326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4caab-fe57-4e7e-89a6-39880956fd22 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:53.512326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4caab-fe57-4e7e-89a6-39880956fd22 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:53.513678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 51/98] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:06:53.519617 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f5646-4505-49f9-aed3-bc8725075ec5 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:53.520017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f5646-4505-49f9-aed3-bc8725075ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:53.520262 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f5646-4505-49f9-aed3-bc8725075ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:53.538840 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-445f5646-4505-49f9-aed3-bc8725075ec5 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:53.540049 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-445f5646-4505-49f9-aed3-bc8725075ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:53.613696 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df416171-c986-4dbd-9e9c-bba2efe2fb3e None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:53.614005 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df416171-c986-4dbd-9e9c-bba2efe2fb3e None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:53.614250 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df416171-c986-4dbd-9e9c-bba2efe2fb3e None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:53.615613 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 52/99] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:06:53.620481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76167fd1-d424-4b3f-8e2d-9d427dcfee4d None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:53.620766 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76167fd1-d424-4b3f-8e2d-9d427dcfee4d None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:53.621043 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76167fd1-d424-4b3f-8e2d-9d427dcfee4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:53.645147 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-76167fd1-d424-4b3f-8e2d-9d427dcfee4d None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:53.646334 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-76167fd1-d424-4b3f-8e2d-9d427dcfee4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:53.654081 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 48/100] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:06:53.694161 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:53.765151 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 53/101] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:06:54.152483 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f5646-4505-49f9-aed3-bc8725075ec5 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:54.274136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f5646-4505-49f9-aed3-bc8725075ec5 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:54.308038 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:54.314588 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d519dc-fcd6-4f38-ab40-cc60c901dc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:54.315265 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d519dc-fcd6-4f38-ab40-cc60c901dc68 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:54.315550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d519dc-fcd6-4f38-ab40-cc60c901dc68 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:54.315823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d519dc-fcd6-4f38-ab40-cc60c901dc68 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:54.328115 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76167fd1-d424-4b3f-8e2d-9d427dcfee4d None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:54.335967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d519dc-fcd6-4f38-ab40-cc60c901dc68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:06:54.340907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-67d519dc-fcd6-4f38-ab40-cc60c901dc68 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:06:54.340907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:06:54.340907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:06:54.340907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:06:54.340907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:06:54.340907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:06:54.340907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:06:54.340907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:06:54.340907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:06:54.340907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:06:54.340907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:06:54.340907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:06:54.340907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:06:54.340907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:06:54.340907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:06:54.340907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:06:54.341824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:06:54.341824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:06:54.341824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:06:54.341824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:06:54.341824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:06:54.341824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:06:54.341824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:06:54.341824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:06:54.341824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:06:54.341824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:06:54.341824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:06:54.341824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:06:54.341824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:06:54.341824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:06:54.341824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:06:54.341824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:06:54.343165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:06:54.343165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:06:54.343165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:06:54.343165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:06:54.343165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:06:54.343165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:06:54.343165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:06:54.343165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:06:54.343165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:06:54.343165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:06:54.343165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:06:54.343165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:06:54.343165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 49/102] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:06:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:06:54.350168 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d519dc-fcd6-4f38-ab40-cc60c901dc68 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:54.513958 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76167fd1-d424-4b3f-8e2d-9d427dcfee4d None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:54.572716 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87bbc26-29d2-4050-9ed6-892e74015f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:54.573371 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bbc26-29d2-4050-9ed6-892e74015f20 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:54.573658 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bbc26-29d2-4050-9ed6-892e74015f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:54.573932 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bbc26-29d2-4050-9ed6-892e74015f20 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:54.577964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87bbc26-29d2-4050-9ed6-892e74015f20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:54.579480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87bbc26-29d2-4050-9ed6-892e74015f20 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:54.581256 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 54/103] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:06:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:54.582627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29e5e8-a51c-44fe-aefa-3207065a444a None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:54.582916 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29e5e8-a51c-44fe-aefa-3207065a444a None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:54.583232 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29e5e8-a51c-44fe-aefa-3207065a444a None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:54.584428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 55/104] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:06:54.589209 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29e5e8-a51c-44fe-aefa-3207065a444a None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:54.593193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d519dc-fcd6-4f38-ab40-cc60c901dc68 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:54.630546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:54.637146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430fb72-a68c-4d19-8dd6-988978b96518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:54.637889 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430fb72-a68c-4d19-8dd6-988978b96518 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:54.638297 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430fb72-a68c-4d19-8dd6-988978b96518 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:54.638668 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430fb72-a68c-4d19-8dd6-988978b96518 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:54.639754 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430fb72-a68c-4d19-8dd6-988978b96518 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:06:54.640312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430fb72-a68c-4d19-8dd6-988978b96518 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:54.641804 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430fb72-a68c-4d19-8dd6-988978b96518 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:54.651067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 50/105] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:06:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:06:54.652182 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf4903-e60f-4841-b4f2-53af37cbca0c None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:54.657436 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf4903-e60f-4841-b4f2-53af37cbca0c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:54.657838 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf4903-e60f-4841-b4f2-53af37cbca0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:54.677438 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-a9cf4903-e60f-4841-b4f2-53af37cbca0c None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:54.678788 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-a9cf4903-e60f-4841-b4f2-53af37cbca0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:54.731212 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29e5e8-a51c-44fe-aefa-3207065a444a None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:54.733434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:54.759171 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 51/106] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:06:54.759171 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c130b11-bb76-4f30-bea9-36d474a38771 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:54.759171 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c130b11-bb76-4f30-bea9-36d474a38771 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:54.759171 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c130b11-bb76-4f30-bea9-36d474a38771 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:54.759171 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 52/107] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:06:54.764765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c130b11-bb76-4f30-bea9-36d474a38771 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:54.804775 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b96a65-28cc-47b7-94a0-bc5cf177f8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:54.805757 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b96a65-28cc-47b7-94a0-bc5cf177f8c5 None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:54.806251 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b96a65-28cc-47b7-94a0-bc5cf177f8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:54.806711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b96a65-28cc-47b7-94a0-bc5cf177f8c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:54.807595 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b96a65-28cc-47b7-94a0-bc5cf177f8c5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:54.809243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b96a65-28cc-47b7-94a0-bc5cf177f8c5 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:54.842579 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 56/108] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 31 00:06:54 2022] POST /identity/v3/projects => generated 302 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:06:54.844545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e2526-3e05-45b2-a6ec-fc8b9eb4873c None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:54.845088 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e2526-3e05-45b2-a6ec-fc8b9eb4873c None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:54.845666 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e2526-3e05-45b2-a6ec-fc8b9eb4873c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:54.869552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-b20e2526-3e05-45b2-a6ec-fc8b9eb4873c None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:54.871145 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b20e2526-3e05-45b2-a6ec-fc8b9eb4873c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:54.962238 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 57/109] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:06:55.013981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c130b11-bb76-4f30-bea9-36d474a38771 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:55.064738 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd940c4-fbb2-4c4c-8afd-1c307a0465ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:55.065319 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd940c4-fbb2-4c4c-8afd-1c307a0465ea None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:55.065605 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd940c4-fbb2-4c4c-8afd-1c307a0465ea None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:55.065886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd940c4-fbb2-4c4c-8afd-1c307a0465ea None admin] PATH_INFO: `/v3/users/35871d61fe1e4c7186aa0cbe990ddba5` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:55.083736 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd940c4-fbb2-4c4c-8afd-1c307a0465ea None admin] RBAC: Authorizing `identity:get_user(user_id=35871d61fe1e4c7186aa0cbe990ddba5)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:55.085229 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd940c4-fbb2-4c4c-8afd-1c307a0465ea None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:55.087260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 53/110] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:06:54 2022] GET /identity/v3/users/35871d61fe1e4c7186aa0cbe990ddba5 => generated 289 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:55.093646 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e2526-3e05-45b2-a6ec-fc8b9eb4873c None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:55.112996 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:55.236661 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e2526-3e05-45b2-a6ec-fc8b9eb4873c None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:55.265073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd940c4-fbb2-4c4c-8afd-1c307a0465ea None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:55.309087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2d0b7-9ce2-4dcc-8b3c-38579368b462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:55.309951 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2d0b7-9ce2-4dcc-8b3c-38579368b462 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:55.310411 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2d0b7-9ce2-4dcc-8b3c-38579368b462 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:55.310862 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2d0b7-9ce2-4dcc-8b3c-38579368b462 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:55.320489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:55.397895 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd940c4-fbb2-4c4c-8afd-1c307a0465ea None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:55.440166 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1330b-ef44-42c0-ae27-a5e1e2874400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:55.441217 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1330b-ef44-42c0-ae27-a5e1e2874400 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:55.441696 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1330b-ef44-42c0-ae27-a5e1e2874400 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:55.442234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1330b-ef44-42c0-ae27-a5e1e2874400 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:55.443573 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2d0b7-9ce2-4dcc-8b3c-38579368b462 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:55.445062 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2d0b7-9ce2-4dcc-8b3c-38579368b462 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:55.454980 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1330b-ef44-42c0-ae27-a5e1e2874400 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:06:55.460146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-6ce1330b-ef44-42c0-ae27-a5e1e2874400 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:06:55.460146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:06:55.460146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:06:55.460146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:06:55.460146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:06:55.460146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:06:55.460146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:06:55.460146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:06:55.460146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:06:55.460146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:06:55.460146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:06:55.460146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:06:55.460146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:06:55.460146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:06:55.460146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:06:55.460146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:06:55.461005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:06:55.461005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:06:55.461005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:06:55.461005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:06:55.461005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:06:55.461005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:06:55.461005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:06:55.461005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:06:55.461005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:06:55.461005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:06:55.461005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:06:55.461005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:06:55.461005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:06:55.461005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:06:55.461005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:06:55.461005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:06:55.461764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:06:55.461764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:06:55.461764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:06:55.461764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:06:55.461764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:06:55.461764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:06:55.461764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:06:55.461764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:06:55.461764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:06:55.461764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:06:55.461764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:06:55.461764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:06:55.462979 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 54/111] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:06:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:06:55.467944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:55.468046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1330b-ef44-42c0-ae27-a5e1e2874400 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:55.559485 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 58/112] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:06:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:06:55.560790 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be0b9b-0a91-4829-b400-318e10d45d23 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:55.561836 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be0b9b-0a91-4829-b400-318e10d45d23 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:55.562182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be0b9b-0a91-4829-b400-318e10d45d23 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:55.568586 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 59/113] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:06:55.573145 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be0b9b-0a91-4829-b400-318e10d45d23 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:55.582158 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:55.585746 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1330b-ef44-42c0-ae27-a5e1e2874400 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:55.626169 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ab8f8-342d-40af-99d6-aacd8e6a5d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:55.627694 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ab8f8-342d-40af-99d6-aacd8e6a5d58 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:55.628153 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ab8f8-342d-40af-99d6-aacd8e6a5d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:55.628672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ab8f8-342d-40af-99d6-aacd8e6a5d58 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:55.640724 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ab8f8-342d-40af-99d6-aacd8e6a5d58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:06:55.646287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-150ab8f8-342d-40af-99d6-aacd8e6a5d58 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:06:55.646287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:06:55.646287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:06:55.646287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:06:55.646287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:06:55.646287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:06:55.646287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:06:55.646287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:06:55.646287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:06:55.646287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:06:55.646287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:06:55.646287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:06:55.646287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:06:55.646287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:06:55.646287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:06:55.646287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:06:55.647151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:06:55.647151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:06:55.647151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:06:55.647151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:06:55.647151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:06:55.647151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:06:55.647151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:06:55.647151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:06:55.647151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:06:55.647151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:06:55.647151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:06:55.647151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:06:55.647151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:06:55.647151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:06:55.647151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:06:55.647151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:06:55.647947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:06:55.647947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:06:55.647947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:06:55.647947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:06:55.647947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:06:55.647947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:06:55.647947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:06:55.647947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:06:55.647947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:06:55.647947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:06:55.647947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:06:55.647947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:06:55.650101 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 55/114] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:06:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:06:55.651063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ab8f8-342d-40af-99d6-aacd8e6a5d58 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:55.688345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be0b9b-0a91-4829-b400-318e10d45d23 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:55.702426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:55.730136 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd644d-843c-43b7-b6cf-ee614ca38ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:55.730766 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd644d-843c-43b7-b6cf-ee614ca38ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:55.731063 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd644d-843c-43b7-b6cf-ee614ca38ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:55.731351 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd644d-843c-43b7-b6cf-ee614ca38ce4 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:55.732034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd644d-843c-43b7-b6cf-ee614ca38ce4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:06:55.732477 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd644d-843c-43b7-b6cf-ee614ca38ce4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:55.734473 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd644d-843c-43b7-b6cf-ee614ca38ce4 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:55.743808 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 60/115] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:06:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:55.745048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5c073-2783-4657-806e-be10e9265dc8 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:55.745344 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5c073-2783-4657-806e-be10e9265dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:55.745626 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5c073-2783-4657-806e-be10e9265dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:55.765692 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-fcf5c073-2783-4657-806e-be10e9265dc8 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:55.766871 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-fcf5c073-2783-4657-806e-be10e9265dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:55.778956 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ab8f8-342d-40af-99d6-aacd8e6a5d58 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:55.819760 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83e025-a53b-426f-ac40-f639279c916a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:55.820397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83e025-a53b-426f-ac40-f639279c916a None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:55.820683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83e025-a53b-426f-ac40-f639279c916a None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:55.820961 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83e025-a53b-426f-ac40-f639279c916a None admin] PATH_INFO: `/v3/projects/9eb64431e703423581c08ff8f8378eb3` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:55.822099 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc83e025-a53b-426f-ac40-f639279c916a None admin] RBAC: Authorizing `identity:get_project(project_id=9eb64431e703423581c08ff8f8378eb3)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:55.823872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc83e025-a53b-426f-ac40-f639279c916a None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:55.830579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 56/116] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:06:55 2022] GET /identity/v3/projects/9eb64431e703423581c08ff8f8378eb3 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:55.831432 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83e025-a53b-426f-ac40-f639279c916a None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:55.836933 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 61/117] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:06:55.844419 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83909206-b3d8-4f87-b0b5-937a248cfc74 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:55.844705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83909206-b3d8-4f87-b0b5-937a248cfc74 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:55.844986 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83909206-b3d8-4f87-b0b5-937a248cfc74 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:55.846445 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 62/118] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:06:55.846974 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83909206-b3d8-4f87-b0b5-937a248cfc74 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:55.866451 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:55.948253 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83e025-a53b-426f-ac40-f639279c916a None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:55.970746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83909206-b3d8-4f87-b0b5-937a248cfc74 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:55.972844 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:56.002502 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c11394-de8a-4b1e-b649-4a56d767b9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:56.003654 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c11394-de8a-4b1e-b649-4a56d767b9bd None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:56.004225 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c11394-de8a-4b1e-b649-4a56d767b9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:56.004745 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c11394-de8a-4b1e-b649-4a56d767b9bd None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:56.005618 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c11394-de8a-4b1e-b649-4a56d767b9bd None admin] RBAC: Adding query filter params (name=member) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:06:56.006296 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c11394-de8a-4b1e-b649-4a56d767b9bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:56.008101 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c11394-de8a-4b1e-b649-4a56d767b9bd None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:56.009023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810f862-9611-4620-b1f7-1bb53760034d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:56.009665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810f862-9611-4620-b1f7-1bb53760034d None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:56.009950 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810f862-9611-4620-b1f7-1bb53760034d None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:56.010229 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810f862-9611-4620-b1f7-1bb53760034d None admin] PATH_INFO: `/v3/users/19ddd6d02b884bcab507cb5ed1181266` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:56.011429 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9810f862-9611-4620-b1f7-1bb53760034d None admin] RBAC: Authorizing `identity:get_user(user_id=19ddd6d02b884bcab507cb5ed1181266)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:56.012895 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9810f862-9611-4620-b1f7-1bb53760034d None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:56.015537 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 63/119] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:06:55 2022] GET /identity/v3/users/19ddd6d02b884bcab507cb5ed1181266 => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:56.019957 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810f862-9611-4620-b1f7-1bb53760034d None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:56.021075 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 57/120] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:06:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:56.027320 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d6adf-be04-4127-b45a-2960139bd229 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:56.027917 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d6adf-be04-4127-b45a-2960139bd229 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:56.028397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d6adf-be04-4127-b45a-2960139bd229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:56.052714 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-0d0d6adf-be04-4127-b45a-2960139bd229 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:56.054974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-0d0d6adf-be04-4127-b45a-2960139bd229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:56.131601 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 58/121] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:06:56.132939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d6adf-be04-4127-b45a-2960139bd229 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:56.146228 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810f862-9611-4620-b1f7-1bb53760034d None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:56.157303 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:56.184631 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168987e-8937-4afb-a099-9b1791bdb965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:56.185664 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168987e-8937-4afb-a099-9b1791bdb965 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:56.186800 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168987e-8937-4afb-a099-9b1791bdb965 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:56.187291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168987e-8937-4afb-a099-9b1791bdb965 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:56.188217 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168987e-8937-4afb-a099-9b1791bdb965 None admin] RBAC: Adding query filter params (role.id=65d36a76676c424db12c090f01595bdc, scope.project.id=9eb64431e703423581c08ff8f8378eb3, user.id=35871d61fe1e4c7186aa0cbe990ddba5) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:06:56.188712 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168987e-8937-4afb-a099-9b1791bdb965 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:56.190588 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168987e-8937-4afb-a099-9b1791bdb965 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:56.197079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 64/122] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:06:56 2022] GET /identity/v3/role_assignments?user.id=35871d61fe1e4c7186aa0cbe990ddba5&scope.project.id=9eb64431e703423581c08ff8f8378eb3&role.id=65d36a76676c424db12c090f01595bdc => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:56.198056 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168987e-8937-4afb-a099-9b1791bdb965 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:56.262779 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:56.304902 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d6adf-be04-4127-b45a-2960139bd229 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:56.347581 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168987e-8937-4afb-a099-9b1791bdb965 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:56.380456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f0293-1861-41bb-bb76-b4f36f4b7e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:56.381126 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f0293-1861-41bb-bb76-b4f36f4b7e47 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:56.381414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f0293-1861-41bb-bb76-b4f36f4b7e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:56.381745 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f0293-1861-41bb-bb76-b4f36f4b7e47 None admin] PATH_INFO: `/v3/projects/9eb64431e703423581c08ff8f8378eb3` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:56.387943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f0293-1861-41bb-bb76-b4f36f4b7e47 None admin] RBAC: Authorizing `identity:get_project(project_id=9eb64431e703423581c08ff8f8378eb3)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:56.390555 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f0293-1861-41bb-bb76-b4f36f4b7e47 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:56.392813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 59/123] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:06:56 2022] GET /identity/v3/projects/9eb64431e703423581c08ff8f8378eb3 => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:56.393691 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f0293-1861-41bb-bb76-b4f36f4b7e47 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:56.399632 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8fbf35-030f-4bc9-a36b-d1b4d45387fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:56.400291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fbf35-030f-4bc9-a36b-d1b4d45387fe None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:56.400573 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fbf35-030f-4bc9-a36b-d1b4d45387fe None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:56.400847 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fbf35-030f-4bc9-a36b-d1b4d45387fe None admin] PATH_INFO: `/v3/users/8eb6244b69ca487281ed852a375261ca` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:56.402015 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8fbf35-030f-4bc9-a36b-d1b4d45387fe None admin] RBAC: Authorizing `identity:get_user(user_id=8eb6244b69ca487281ed852a375261ca)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:56.403680 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8fbf35-030f-4bc9-a36b-d1b4d45387fe None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:56.407431 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 65/124] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:06:56 2022] GET /identity/v3/users/8eb6244b69ca487281ed852a375261ca => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:56.408264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8fbf35-030f-4bc9-a36b-d1b4d45387fe None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:56.537414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:56.551487 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f0293-1861-41bb-bb76-b4f36f4b7e47 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:56.558970 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8fbf35-030f-4bc9-a36b-d1b4d45387fe None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:56.563345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:56.587502 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d2245-6a00-45ce-a638-27b8164af22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:56.588232 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d2245-6a00-45ce-a638-27b8164af22b None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:56.588413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d2245-6a00-45ce-a638-27b8164af22b None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:56.588684 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d2245-6a00-45ce-a638-27b8164af22b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:56.598198 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b470f-604f-4886-b79c-eaf71f5a8479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:56.599110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b470f-604f-4886-b79c-eaf71f5a8479 None admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:56.599535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b470f-604f-4886-b79c-eaf71f5a8479 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:56.599958 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b470f-604f-4886-b79c-eaf71f5a8479 None admin] PATH_INFO: `/v3/projects/9eb64431e703423581c08ff8f8378eb3/users/19ddd6d02b884bcab507cb5ed1181266/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:56.600068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d2245-6a00-45ce-a638-27b8164af22b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:06:56.602347 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b470f-604f-4886-b79c-eaf71f5a8479 None admin] RBAC: Authorizing `identity:create_grant(project_id=9eb64431e703423581c08ff8f8378eb3, user_id=19ddd6d02b884bcab507cb5ed1181266, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:56.604183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b470f-604f-4886-b79c-eaf71f5a8479 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:56.605274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-c04d2245-6a00-45ce-a638-27b8164af22b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:06:56.605274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:06:56.605274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:06:56.605274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:06:56.605274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:06:56.605274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:06:56.605274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:06:56.605274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:06:56.605274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:06:56.605274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:06:56.605274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:06:56.605274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:06:56.605274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:06:56.605274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:06:56.605274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:06:56.605274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:06:56.606284 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:06:56.606284 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:06:56.606284 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:06:56.606284 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:06:56.606284 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:06:56.606284 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:06:56.606284 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:06:56.606284 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:06:56.606284 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:06:56.606284 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:06:56.606284 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:06:56.606284 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:06:56.606284 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:06:56.606284 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:06:56.606284 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:06:56.606284 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:06:56.607193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:06:56.607193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:06:56.607193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:06:56.607193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:06:56.607193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:06:56.607193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:06:56.607193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:06:56.607193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:06:56.607193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:06:56.607193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:06:56.607193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:06:56.607193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:06:56.607193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 60/125] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 31 00:06:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:06:56.612586 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d2245-6a00-45ce-a638-27b8164af22b None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:56.625047 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 66/126] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 31 00:06:56 2022] PUT /identity/v3/projects/9eb64431e703423581c08ff8f8378eb3/users/19ddd6d02b884bcab507cb5ed1181266/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:06:56.633212 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b470f-604f-4886-b79c-eaf71f5a8479 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:56.764412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d2245-6a00-45ce-a638-27b8164af22b None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:56.809867 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a35cc7-2425-40cd-966c-46b81857f3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:56.811498 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a35cc7-2425-40cd-966c-46b81857f3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:56.811786 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a35cc7-2425-40cd-966c-46b81857f3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:56.812062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a35cc7-2425-40cd-966c-46b81857f3c8 None admin] PATH_INFO: `/v3/projects/9eb64431e703423581c08ff8f8378eb3` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:56.813896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a35cc7-2425-40cd-966c-46b81857f3c8 None admin] RBAC: Authorizing `identity:get_project(project_id=9eb64431e703423581c08ff8f8378eb3)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:56.815984 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a35cc7-2425-40cd-966c-46b81857f3c8 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:56.818105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 61/127] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:06:56 2022] GET /identity/v3/projects/9eb64431e703423581c08ff8f8378eb3 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:56.819354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a35cc7-2425-40cd-966c-46b81857f3c8 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:56.820972 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b470f-604f-4886-b79c-eaf71f5a8479 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:56.842463 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:56.875931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa351ab1-e908-42af-99b2-e4f3f6d181e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:56.876619 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa351ab1-e908-42af-99b2-e4f3f6d181e7 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:56.876903 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa351ab1-e908-42af-99b2-e4f3f6d181e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:56.877184 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa351ab1-e908-42af-99b2-e4f3f6d181e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:56.890528 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa351ab1-e908-42af-99b2-e4f3f6d181e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:06:56.897110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-aa351ab1-e908-42af-99b2-e4f3f6d181e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:06:56.897110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:06:56.897110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:06:56.897110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:06:56.897110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:06:56.897110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:06:56.897110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:06:56.897110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:06:56.897110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:06:56.897110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:06:56.897110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:06:56.897110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:06:56.897110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:06:56.897110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:06:56.897110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:06:56.897110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:06:56.897965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:06:56.897965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:06:56.897965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:06:56.897965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:06:56.897965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:06:56.897965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:06:56.897965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:06:56.897965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:06:56.897965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:06:56.897965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:06:56.897965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:06:56.897965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:06:56.897965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:06:56.897965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:06:56.897965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:06:56.897965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:06:56.898713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:06:56.898713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:06:56.898713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:06:56.898713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:06:56.898713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:06:56.898713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:06:56.898713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:06:56.898713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:06:56.898713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:06:56.898713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:06:56.898713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:06:56.898713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:06:56.898713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 67/128] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:06:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:06:56.899463 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa351ab1-e908-42af-99b2-e4f3f6d181e7 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:56.943439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:56.948338 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a35cc7-2425-40cd-966c-46b81857f3c8 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:56.992757 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba8227-b7f3-40e0-9a93-cb9bf66f6424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:56.992757 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba8227-b7f3-40e0-9a93-cb9bf66f6424 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:56.993027 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba8227-b7f3-40e0-9a93-cb9bf66f6424 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:56.994169 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba8227-b7f3-40e0-9a93-cb9bf66f6424 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:56.994169 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba8227-b7f3-40e0-9a93-cb9bf66f6424 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:06:56.994169 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba8227-b7f3-40e0-9a93-cb9bf66f6424 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:56.995822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba8227-b7f3-40e0-9a93-cb9bf66f6424 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:57.008250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 62/129] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:06:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:57.009106 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba8227-b7f3-40e0-9a93-cb9bf66f6424 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:57.067659 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa351ab1-e908-42af-99b2-e4f3f6d181e7 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:57.111453 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:57.133193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba8227-b7f3-40e0-9a93-cb9bf66f6424 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:57.153050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502dd84-abb2-480f-ab18-74393d7da4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:57.153653 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502dd84-abb2-480f-ab18-74393d7da4e5 None admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:57.153935 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502dd84-abb2-480f-ab18-74393d7da4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:57.156438 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502dd84-abb2-480f-ab18-74393d7da4e5 None admin] PATH_INFO: `/v3/projects/9eb64431e703423581c08ff8f8378eb3/users/8eb6244b69ca487281ed852a375261ca/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:57.162476 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3502dd84-abb2-480f-ab18-74393d7da4e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=9eb64431e703423581c08ff8f8378eb3, user_id=8eb6244b69ca487281ed852a375261ca, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:57.163979 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3502dd84-abb2-480f-ab18-74393d7da4e5 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:57.182738 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0ff4a-6357-48a3-afbd-b82f1c673fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:57.183382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0ff4a-6357-48a3-afbd-b82f1c673fb3 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:57.183797 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0ff4a-6357-48a3-afbd-b82f1c673fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:57.184072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0ff4a-6357-48a3-afbd-b82f1c673fb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:57.184745 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac0ff4a-6357-48a3-afbd-b82f1c673fb3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:06:57.185223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac0ff4a-6357-48a3-afbd-b82f1c673fb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:57.194215 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 68/130] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 31 00:06:56 2022] PUT /identity/v3/projects/9eb64431e703423581c08ff8f8378eb3/users/8eb6244b69ca487281ed852a375261ca/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:06:57.195195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502dd84-abb2-480f-ab18-74393d7da4e5 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:57.196020 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac0ff4a-6357-48a3-afbd-b82f1c673fb3 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:57.213723 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 63/131] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:06:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:57.222198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0ff4a-6357-48a3-afbd-b82f1c673fb3 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:57.359739 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:57.378718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0ff4a-6357-48a3-afbd-b82f1c673fb3 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:57.413405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502dd84-abb2-480f-ab18-74393d7da4e5 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:57.416624 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc1c418-fe93-4333-aa97-1110e0fe1247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:57.417527 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1c418-fe93-4333-aa97-1110e0fe1247 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:57.418034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1c418-fe93-4333-aa97-1110e0fe1247 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:57.418470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1c418-fe93-4333-aa97-1110e0fe1247 None admin] PATH_INFO: `/v3/users/8eb6244b69ca487281ed852a375261ca` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:57.419843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc1c418-fe93-4333-aa97-1110e0fe1247 None admin] RBAC: Authorizing `identity:get_user(user_id=8eb6244b69ca487281ed852a375261ca)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:57.421719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc1c418-fe93-4333-aa97-1110e0fe1247 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:57.423670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 64/132] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:06:57 2022] GET /identity/v3/users/8eb6244b69ca487281ed852a375261ca => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:57.429842 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc1c418-fe93-4333-aa97-1110e0fe1247 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:57.444974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:57.469951 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a84f6-ab57-411d-a2d0-b0735361f6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:57.470836 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a84f6-ab57-411d-a2d0-b0735361f6bc None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:57.471311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a84f6-ab57-411d-a2d0-b0735361f6bc None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:57.471709 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a84f6-ab57-411d-a2d0-b0735361f6bc None admin] PATH_INFO: `/v3/users/8eb6244b69ca487281ed852a375261ca` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:57.473266 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56a84f6-ab57-411d-a2d0-b0735361f6bc None admin] RBAC: Authorizing `identity:get_user(user_id=8eb6244b69ca487281ed852a375261ca)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:57.475028 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56a84f6-ab57-411d-a2d0-b0735361f6bc None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:57.477662 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 69/133] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:06:57 2022] GET /identity/v3/users/8eb6244b69ca487281ed852a375261ca => generated 275 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:57.479157 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7ce35-fe6f-44e5-b9cc-b28f436312ab None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:57.479742 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7ce35-fe6f-44e5-b9cc-b28f436312ab None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:57.480296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7ce35-fe6f-44e5-b9cc-b28f436312ab None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:57.482026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 70/134] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:06:57.487936 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a665de1-b666-42b4-9e67-37d110e23fcf None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:57.488480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a665de1-b666-42b4-9e67-37d110e23fcf None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:57.488943 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a665de1-b666-42b4-9e67-37d110e23fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:57.521507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-4a665de1-b666-42b4-9e67-37d110e23fcf None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:57.521507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4a665de1-b666-42b4-9e67-37d110e23fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:57.574237 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc1c418-fe93-4333-aa97-1110e0fe1247 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:57.596578 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 71/135] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:06:57.597978 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a665de1-b666-42b4-9e67-37d110e23fcf None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:57.613380 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be86e77c-60b1-425c-9369-d5f46ceb24f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:57.614169 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86e77c-60b1-425c-9369-d5f46ceb24f9 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:57.614834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86e77c-60b1-425c-9369-d5f46ceb24f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:57.614834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86e77c-60b1-425c-9369-d5f46ceb24f9 None admin] PATH_INFO: `/v3/projects/be6cea0351cb401184e319eef387800a` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:57.616187 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be86e77c-60b1-425c-9369-d5f46ceb24f9 None admin] RBAC: Authorizing `identity:get_project(project_id=be6cea0351cb401184e319eef387800a)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:57.617831 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be86e77c-60b1-425c-9369-d5f46ceb24f9 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:57.624931 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 65/136] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:06:57 2022] GET /identity/v3/projects/be6cea0351cb401184e319eef387800a => generated 312 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:57.626413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be86e77c-60b1-425c-9369-d5f46ceb24f9 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:57.684318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:57.705453 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a665de1-b666-42b4-9e67-37d110e23fcf None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:57.714902 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:57.742212 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be86e77c-60b1-425c-9369-d5f46ceb24f9 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:57.752939 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05a9f8-2fbf-4032-bfcd-7dad35c8170a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:57.753735 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05a9f8-2fbf-4032-bfcd-7dad35c8170a None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:57.753914 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05a9f8-2fbf-4032-bfcd-7dad35c8170a None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:57.754193 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05a9f8-2fbf-4032-bfcd-7dad35c8170a None admin] PATH_INFO: `/v3/projects/9eb64431e703423581c08ff8f8378eb3` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:57.763463 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05a9f8-2fbf-4032-bfcd-7dad35c8170a None admin] RBAC: Authorizing `identity:get_project(project_id=9eb64431e703423581c08ff8f8378eb3)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:57.765792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05a9f8-2fbf-4032-bfcd-7dad35c8170a None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:57.767495 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 72/137] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:06:57 2022] GET /identity/v3/projects/9eb64431e703423581c08ff8f8378eb3 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:57.773184 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05a9f8-2fbf-4032-bfcd-7dad35c8170a None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:57.782635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818b66a-85ce-43a7-81b6-fe5ae397649b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:57.784310 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818b66a-85ce-43a7-81b6-fe5ae397649b None admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:57.784783 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818b66a-85ce-43a7-81b6-fe5ae397649b None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:57.785551 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818b66a-85ce-43a7-81b6-fe5ae397649b None admin] PATH_INFO: `/v3/projects/be6cea0351cb401184e319eef387800a/users/8eb6244b69ca487281ed852a375261ca/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:57.793996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818b66a-85ce-43a7-81b6-fe5ae397649b None admin] RBAC: Authorizing `identity:create_grant(project_id=be6cea0351cb401184e319eef387800a, user_id=8eb6244b69ca487281ed852a375261ca, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:57.796323 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818b66a-85ce-43a7-81b6-fe5ae397649b None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:57.824913 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 66/138] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 31 00:06:57 2022] PUT /identity/v3/projects/be6cea0351cb401184e319eef387800a/users/8eb6244b69ca487281ed852a375261ca/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:06:57.922526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05a9f8-2fbf-4032-bfcd-7dad35c8170a None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:57.961534 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2417029-9afb-4565-bdff-055c76677820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:57.962525 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2417029-9afb-4565-bdff-055c76677820 None admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:57.963011 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2417029-9afb-4565-bdff-055c76677820 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:57.963510 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2417029-9afb-4565-bdff-055c76677820 None admin] PATH_INFO: `/v3/projects/9eb64431e703423581c08ff8f8378eb3/users/8eb6244b69ca487281ed852a375261ca/roles/4d91c00f40304b9e9ef4e6c89f46e5c5` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:57.965921 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2417029-9afb-4565-bdff-055c76677820 None admin] RBAC: Authorizing `identity:create_grant(project_id=9eb64431e703423581c08ff8f8378eb3, user_id=8eb6244b69ca487281ed852a375261ca, role_id=4d91c00f40304b9e9ef4e6c89f46e5c5)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:57.968492 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2417029-9afb-4565-bdff-055c76677820 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:58.007710 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 73/139] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 31 00:06:57 2022] PUT /identity/v3/projects/9eb64431e703423581c08ff8f8378eb3/users/8eb6244b69ca487281ed852a375261ca/roles/4d91c00f40304b9e9ef4e6c89f46e5c5 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:06:58.095562 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818b66a-85ce-43a7-81b6-fe5ae397649b None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:58.222479 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818b66a-85ce-43a7-81b6-fe5ae397649b None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:58.265519 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebeb1a8-d1c9-454f-b322-21d8884e46c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:58.266173 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebeb1a8-d1c9-454f-b322-21d8884e46c9 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:58.266455 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebeb1a8-d1c9-454f-b322-21d8884e46c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:58.266734 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebeb1a8-d1c9-454f-b322-21d8884e46c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:58.267990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebeb1a8-d1c9-454f-b322-21d8884e46c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:58.269384 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebeb1a8-d1c9-454f-b322-21d8884e46c9 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:58.271060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 67/140] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:06:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:58.276591 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2417029-9afb-4565-bdff-055c76677820 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:58.288633 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:58.387718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2417029-9afb-4565-bdff-055c76677820 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:58.432306 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c79dff-da63-437d-9956-cb09088a66a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:58.432927 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c79dff-da63-437d-9956-cb09088a66a5 None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:58.433209 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c79dff-da63-437d-9956-cb09088a66a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:58.433480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c79dff-da63-437d-9956-cb09088a66a5 None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:58.434153 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c79dff-da63-437d-9956-cb09088a66a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:58.436298 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c79dff-da63-437d-9956-cb09088a66a5 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:58.474939 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 74/141] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 31 00:06:58 2022] POST /identity/v3/users => generated 283 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:06:58.759503 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ec034-e8bb-40df-bfd6-132258c471da None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:58.759831 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ec034-e8bb-40df-bfd6-132258c471da None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:58.760002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ec034-e8bb-40df-bfd6-132258c471da None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:58.761333 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 68/142] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:06:58.769011 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4d472-a02a-483b-8366-81dfe72c85bb None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:58.769463 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4d472-a02a-483b-8366-81dfe72c85bb None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:58.769659 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4d472-a02a-483b-8366-81dfe72c85bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:58.800079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-86b4d472-a02a-483b-8366-81dfe72c85bb None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:58.801294 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-86b4d472-a02a-483b-8366-81dfe72c85bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:58.880133 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:58.895213 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 75/143] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:06:59.179725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ec034-e8bb-40df-bfd6-132258c471da None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:59.183955 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:59.331165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ec034-e8bb-40df-bfd6-132258c471da None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:59.367939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e18f4-202a-44e3-81af-d1a150be0ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:59.368583 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e18f4-202a-44e3-81af-d1a150be0ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:59.368871 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e18f4-202a-44e3-81af-d1a150be0ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:59.369251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e18f4-202a-44e3-81af-d1a150be0ab2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:59.388080 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e18f4-202a-44e3-81af-d1a150be0ab2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:06:59.393988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-8e3e18f4-202a-44e3-81af-d1a150be0ab2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:06:59.393988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:06:59.393988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:06:59.393988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:06:59.393988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:06:59.393988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:06:59.393988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:06:59.393988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:06:59.393988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:06:59.393988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:06:59.393988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:06:59.393988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:06:59.393988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:06:59.393988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:06:59.393988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:06:59.393988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:06:59.394850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:06:59.394850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:06:59.394850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:06:59.394850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:06:59.394850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:06:59.394850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:06:59.394850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:06:59.394850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:06:59.394850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:06:59.394850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:06:59.394850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:06:59.394850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:06:59.394850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:06:59.394850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:06:59.394850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:06:59.394850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:06:59.396091 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:06:59.396091 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:06:59.396091 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:06:59.396091 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:06:59.396091 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:06:59.396091 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:06:59.396091 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:06:59.396091 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:06:59.396091 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:06:59.396091 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:06:59.396091 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:06:59.396091 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:06:59.396091 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 69/144] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:06:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:06:59.403188 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4d472-a02a-483b-8366-81dfe72c85bb None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:59.550436 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4d472-a02a-483b-8366-81dfe72c85bb None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:59.586992 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280082dc-b234-4832-becd-da8def31e030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:59.587998 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280082dc-b234-4832-becd-da8def31e030 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:59.588468 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280082dc-b234-4832-becd-da8def31e030 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:59.588933 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280082dc-b234-4832-becd-da8def31e030 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:59.589799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280082dc-b234-4832-becd-da8def31e030 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:06:59.590301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280082dc-b234-4832-becd-da8def31e030 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:59.592473 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280082dc-b234-4832-becd-da8def31e030 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:59.607012 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 76/145] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:06:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:59.613208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e18f4-202a-44e3-81af-d1a150be0ab2 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:59.686938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31eb8bc-4aa2-44bb-848a-6e93bb7f5928 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:59.687610 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31eb8bc-4aa2-44bb-848a-6e93bb7f5928 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:59.688369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31eb8bc-4aa2-44bb-848a-6e93bb7f5928 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:59.689936 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 77/146] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:06:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:06:59.695956 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4514dc-4b99-443e-9d3f-c46917e363ff None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:59.696717 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4514dc-4b99-443e-9d3f-c46917e363ff None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:59.697204 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4514dc-4b99-443e-9d3f-c46917e363ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:59.698948 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:59.734173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-8d4514dc-4b99-443e-9d3f-c46917e363ff None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:06:59.735751 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-8d4514dc-4b99-443e-9d3f-c46917e363ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:06:59.765496 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e18f4-202a-44e3-81af-d1a150be0ab2 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:59.831401 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:06:59.833281 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e1fe9-4214-4469-8497-d1847cdf518e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:06:59.833944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e1fe9-4214-4469-8497-d1847cdf518e None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:06:59.834235 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e1fe9-4214-4469-8497-d1847cdf518e None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:06:59.834567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e1fe9-4214-4469-8497-d1847cdf518e None admin] PATH_INFO: `/v3/users/35871d61fe1e4c7186aa0cbe990ddba5` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:06:59.836714 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e1fe9-4214-4469-8497-d1847cdf518e None admin] RBAC: Authorizing `identity:get_user(user_id=35871d61fe1e4c7186aa0cbe990ddba5)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:06:59.838993 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e1fe9-4214-4469-8497-d1847cdf518e None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:06:59.841897 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 70/147] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:06:59 2022] GET /identity/v3/users/35871d61fe1e4c7186aa0cbe990ddba5 => generated 289 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:06:59.847689 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e1fe9-4214-4469-8497-d1847cdf518e None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:06:59.862734 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 78/148] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:06:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:06:59.963994 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e1fe9-4214-4469-8497-d1847cdf518e None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:06:59.995701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:00.010139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791bbc5-8f3a-4f01-aa7c-3350d87c3ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:00.011154 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791bbc5-8f3a-4f01-aa7c-3350d87c3ded None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:00.011628 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791bbc5-8f3a-4f01-aa7c-3350d87c3ded None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:00.012120 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791bbc5-8f3a-4f01-aa7c-3350d87c3ded None admin] PATH_INFO: `/v3/projects/9eb64431e703423581c08ff8f8378eb3` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:00.019776 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791bbc5-8f3a-4f01-aa7c-3350d87c3ded None admin] RBAC: Authorizing `identity:get_project(project_id=9eb64431e703423581c08ff8f8378eb3)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:00.021519 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791bbc5-8f3a-4f01-aa7c-3350d87c3ded None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:00.023461 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 71/149] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:06:59 2022] GET /identity/v3/projects/9eb64431e703423581c08ff8f8378eb3 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:00.030627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4514dc-4b99-443e-9d3f-c46917e363ff None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:00.167481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4514dc-4b99-443e-9d3f-c46917e363ff None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:00.211867 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30acf4b2-6dae-4b0a-a529-5ba4693677ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:00.212540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acf4b2-6dae-4b0a-a529-5ba4693677ed None admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:00.212822 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acf4b2-6dae-4b0a-a529-5ba4693677ed None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:00.213096 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acf4b2-6dae-4b0a-a529-5ba4693677ed None admin] PATH_INFO: `/v3/projects/9eb64431e703423581c08ff8f8378eb3/users/35871d61fe1e4c7186aa0cbe990ddba5/roles/65d36a76676c424db12c090f01595bdc` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:00.215064 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30acf4b2-6dae-4b0a-a529-5ba4693677ed None admin] RBAC: Authorizing `identity:create_grant(project_id=9eb64431e703423581c08ff8f8378eb3, user_id=35871d61fe1e4c7186aa0cbe990ddba5, role_id=65d36a76676c424db12c090f01595bdc)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:00.216518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30acf4b2-6dae-4b0a-a529-5ba4693677ed None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:00.240472 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 79/150] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 31 00:07:00 2022] PUT /identity/v3/projects/9eb64431e703423581c08ff8f8378eb3/users/35871d61fe1e4c7186aa0cbe990ddba5/roles/65d36a76676c424db12c090f01595bdc => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:07:00.509122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791bbc5-8f3a-4f01-aa7c-3350d87c3ded None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:00.562038 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b426363-d136-4d75-b109-9301f01776d8 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:00.562375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b426363-d136-4d75-b109-9301f01776d8 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:00.562595 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b426363-d136-4d75-b109-9301f01776d8 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:00.563990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 80/151] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:00.569506 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8bc02-618f-4286-825b-da072829e7c6 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:00.569791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8bc02-618f-4286-825b-da072829e7c6 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:00.570069 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8bc02-618f-4286-825b-da072829e7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:00.603863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-79c8bc02-618f-4286-825b-da072829e7c6 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:00.605078 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-79c8bc02-618f-4286-825b-da072829e7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:00.707394 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:00.709004 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791bbc5-8f3a-4f01-aa7c-3350d87c3ded None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:00.761102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 81/152] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:07:00.775893 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44905691-4b5d-4d64-ab83-f4795cbf2389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:00.776552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44905691-4b5d-4d64-ab83-f4795cbf2389 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:00.776836 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44905691-4b5d-4d64-ab83-f4795cbf2389 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:00.777115 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44905691-4b5d-4d64-ab83-f4795cbf2389 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:00.797132 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44905691-4b5d-4d64-ab83-f4795cbf2389 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:00.808707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-44905691-4b5d-4d64-ab83-f4795cbf2389 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:07:00.808707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:00.808707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:00.808707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:00.808707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:00.808707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:00.808707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:00.808707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:00.808707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:00.808707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:00.808707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:00.808707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:00.808707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:00.808707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:00.808707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:00.808707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:00.809763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:00.809763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:00.809763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:00.809763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:00.809763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:00.809763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:00.809763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:00.809763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:00.809763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:00.809763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:00.809763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:00.809763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:00.809763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:00.809763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:00.809763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:00.809763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:00.811213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:00.811213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:00.811213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:00.811213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:00.811213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:00.811213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:00.811213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:00.811213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:00.811213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:00.811213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:00.811213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:07:00.811213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:00.813720 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 72/153] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:07:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:00.819209 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8bc02-618f-4286-825b-da072829e7c6 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:00.956361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8bc02-618f-4286-825b-da072829e7c6 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:01.009948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19556ed9-9154-4604-af4e-8127a93a0786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:01.010639 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19556ed9-9154-4604-af4e-8127a93a0786 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:01.010899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19556ed9-9154-4604-af4e-8127a93a0786 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:01.011251 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19556ed9-9154-4604-af4e-8127a93a0786 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:01.011917 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19556ed9-9154-4604-af4e-8127a93a0786 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:01.012221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19556ed9-9154-4604-af4e-8127a93a0786 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:01.013623 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19556ed9-9154-4604-af4e-8127a93a0786 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:01.025864 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 82/154] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:01.032205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44905691-4b5d-4d64-ab83-f4795cbf2389 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:01.119659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:01.171016 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44905691-4b5d-4d64-ab83-f4795cbf2389 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:01.185955 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19556ed9-9154-4604-af4e-8127a93a0786 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:01.210648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fca5de-c46d-4784-b3fe-589bcc03f8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:01.215061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fca5de-c46d-4784-b3fe-589bcc03f8f1 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:01.215061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fca5de-c46d-4784-b3fe-589bcc03f8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:01.215061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fca5de-c46d-4784-b3fe-589bcc03f8f1 None admin] PATH_INFO: `/v3/users/19ddd6d02b884bcab507cb5ed1181266` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:01.215061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fca5de-c46d-4784-b3fe-589bcc03f8f1 None admin] RBAC: Authorizing `identity:get_user(user_id=19ddd6d02b884bcab507cb5ed1181266)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:01.215061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fca5de-c46d-4784-b3fe-589bcc03f8f1 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:01.216944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 73/155] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:07:01 2022] GET /identity/v3/users/19ddd6d02b884bcab507cb5ed1181266 => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:01.222695 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fca5de-c46d-4784-b3fe-589bcc03f8f1 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:01.280409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:01.387324 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fca5de-c46d-4784-b3fe-589bcc03f8f1 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:01.410929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19556ed9-9154-4604-af4e-8127a93a0786 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:01.435393 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:01.450817 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249183fe-d812-4024-93e2-4f6466d8d1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:01.452057 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249183fe-d812-4024-93e2-4f6466d8d1d6 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:01.452370 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249183fe-d812-4024-93e2-4f6466d8d1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:01.452714 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249183fe-d812-4024-93e2-4f6466d8d1d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:01.473494 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db5a03-126b-4fde-81d9-7eaff84551a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:01.474141 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db5a03-126b-4fde-81d9-7eaff84551a4 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:01.474419 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db5a03-126b-4fde-81d9-7eaff84551a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:01.474695 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db5a03-126b-4fde-81d9-7eaff84551a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:01.499960 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db5a03-126b-4fde-81d9-7eaff84551a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:01.508985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-23db5a03-126b-4fde-81d9-7eaff84551a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:01.508985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:01.508985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:01.508985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:01.508985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:01.508985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:01.508985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:01.508985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:01.508985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:01.508985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:01.508985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:01.508985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:01.508985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:01.508985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:01.508985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:01.508985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:01.510059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:01.510059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:01.510059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:01.510059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:01.510059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:01.510059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:01.510059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:01.510059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:01.510059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:01.510059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:01.510059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:01.510059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:01.510059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:01.510059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:01.510059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:01.510059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:01.510910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:01.510910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:01.510910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:01.510910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:01.510910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:01.510910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:01.510910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:01.510910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:01.510910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:01.510910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:01.510910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:01.510910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:01.510910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 83/156] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:07:01.512881 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ed83b-5ab8-457d-97c5-3e24c7dbf55e None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:01.512881 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ed83b-5ab8-457d-97c5-3e24c7dbf55e None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:01.512881 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ed83b-5ab8-457d-97c5-3e24c7dbf55e None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:01.520716 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 84/157] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:01.529859 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ef91a-0234-4d0d-9e52-80ad92adf421 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:01.530365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ef91a-0234-4d0d-9e52-80ad92adf421 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:01.530713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ef91a-0234-4d0d-9e52-80ad92adf421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:01.579269 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-718ef91a-0234-4d0d-9e52-80ad92adf421 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:01.580678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-718ef91a-0234-4d0d-9e52-80ad92adf421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:01.605911 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249183fe-d812-4024-93e2-4f6466d8d1d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:01.607685 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249183fe-d812-4024-93e2-4f6466d8d1d6 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:01.705836 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 85/158] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:01.707148 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ef91a-0234-4d0d-9e52-80ad92adf421 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:01.738398 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:01.772675 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 74/159] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:01.774078 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a2d21b-718a-472c-adaa-b506ab55b532 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:01.774369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a2d21b-718a-472c-adaa-b506ab55b532 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:01.774645 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a2d21b-718a-472c-adaa-b506ab55b532 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:01.783530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 75/160] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:01 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:01.784180 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a2d21b-718a-472c-adaa-b506ab55b532 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:01.829685 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:01.830852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ef91a-0234-4d0d-9e52-80ad92adf421 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:01.867827 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5cfa1a-1379-4f2c-b8cb-dba3adcd6a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:01.868811 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cfa1a-1379-4f2c-b8cb-dba3adcd6a87 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:01.869312 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cfa1a-1379-4f2c-b8cb-dba3adcd6a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:01.869781 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cfa1a-1379-4f2c-b8cb-dba3adcd6a87 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:01.870666 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5cfa1a-1379-4f2c-b8cb-dba3adcd6a87 None admin] RBAC: Adding query filter params (name=member) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:01.871213 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5cfa1a-1379-4f2c-b8cb-dba3adcd6a87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:01.872806 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5cfa1a-1379-4f2c-b8cb-dba3adcd6a87 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:01.882960 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 86/161] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:07:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:01.884670 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac5308-bd6a-4bd0-b9ca-68bb707e7319 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:01.885171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac5308-bd6a-4bd0-b9ca-68bb707e7319 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:01.885654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac5308-bd6a-4bd0-b9ca-68bb707e7319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:01.906006 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-3dac5308-bd6a-4bd0-b9ca-68bb707e7319 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:01.907587 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-3dac5308-bd6a-4bd0-b9ca-68bb707e7319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:01.965927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a2d21b-718a-472c-adaa-b506ab55b532 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:02.044460 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 87/162] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:02.045991 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac5308-bd6a-4bd0-b9ca-68bb707e7319 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:02.046776 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1043de92-4a21-40a3-9167-0511d884d104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:02.047588 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043de92-4a21-40a3-9167-0511d884d104 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:02.047967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043de92-4a21-40a3-9167-0511d884d104 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:02.048372 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043de92-4a21-40a3-9167-0511d884d104 None admin] PATH_INFO: `/v3/projects/9eb64431e703423581c08ff8f8378eb3` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:02.059524 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1043de92-4a21-40a3-9167-0511d884d104 None admin] RBAC: Authorizing `identity:get_project(project_id=9eb64431e703423581c08ff8f8378eb3)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:02.059524 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1043de92-4a21-40a3-9167-0511d884d104 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:02.059524 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 76/163] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:01 2022] GET /identity/v3/projects/9eb64431e703423581c08ff8f8378eb3 => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:02.082208 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:02.083168 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1043de92-4a21-40a3-9167-0511d884d104 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:02.176883 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac5308-bd6a-4bd0-b9ca-68bb707e7319 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:02.216811 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc94c1-7d82-454f-b287-cd4355c2af63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:02.217865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc94c1-7d82-454f-b287-cd4355c2af63 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:02.218339 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc94c1-7d82-454f-b287-cd4355c2af63 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:02.218753 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc94c1-7d82-454f-b287-cd4355c2af63 None admin] PATH_INFO: `/v3/users/8eb6244b69ca487281ed852a375261ca` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:02.220101 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cc94c1-7d82-454f-b287-cd4355c2af63 None admin] RBAC: Authorizing `identity:get_user(user_id=8eb6244b69ca487281ed852a375261ca)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:02.221709 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cc94c1-7d82-454f-b287-cd4355c2af63 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:02.223466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 88/164] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:07:02 2022] GET /identity/v3/users/8eb6244b69ca487281ed852a375261ca => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:02.230074 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc94c1-7d82-454f-b287-cd4355c2af63 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:02.254460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:02.264567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1043de92-4a21-40a3-9167-0511d884d104 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:02.334153 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd7387-26e2-4ca0-9fb1-04f6145961af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:02.334990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd7387-26e2-4ca0-9fb1-04f6145961af None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:02.336851 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd7387-26e2-4ca0-9fb1-04f6145961af None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:02.337266 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd7387-26e2-4ca0-9fb1-04f6145961af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:02.339163 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fd7387-26e2-4ca0-9fb1-04f6145961af None admin] RBAC: Adding query filter params (role.id=34df06ce84ad4cf8b3bd1a4490228017, scope.project.id=9eb64431e703423581c08ff8f8378eb3, user.id=19ddd6d02b884bcab507cb5ed1181266) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:02.339652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fd7387-26e2-4ca0-9fb1-04f6145961af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:02.342501 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fd7387-26e2-4ca0-9fb1-04f6145961af None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:02.357853 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 77/165] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:07:02 2022] GET /identity/v3/role_assignments?user.id=19ddd6d02b884bcab507cb5ed1181266&scope.project.id=9eb64431e703423581c08ff8f8378eb3&role.id=34df06ce84ad4cf8b3bd1a4490228017 => generated 608 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:02.358904 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd7387-26e2-4ca0-9fb1-04f6145961af None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:02.411062 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:02.442786 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc94c1-7d82-454f-b287-cd4355c2af63 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:02.512714 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb76c8-1a1b-490b-beee-867e891cf3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:02.514232 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb76c8-1a1b-490b-beee-867e891cf3e8 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:02.514945 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb76c8-1a1b-490b-beee-867e891cf3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:02.515393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb76c8-1a1b-490b-beee-867e891cf3e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:02.589138 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd7387-26e2-4ca0-9fb1-04f6145961af None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:02.621223 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb76c8-1a1b-490b-beee-867e891cf3e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:02.628751 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb76c8-1a1b-490b-beee-867e891cf3e8 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:02.641215 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633701a7-99be-41a6-9e4d-421bc6fd171d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:02.641926 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633701a7-99be-41a6-9e4d-421bc6fd171d None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:02.642213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633701a7-99be-41a6-9e4d-421bc6fd171d None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:02.642488 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633701a7-99be-41a6-9e4d-421bc6fd171d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:02.674501 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633701a7-99be-41a6-9e4d-421bc6fd171d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:02.680204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-633701a7-99be-41a6-9e4d-421bc6fd171d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:07:02.680204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:02.680204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:02.680204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:02.680204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:02.680204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:02.680204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:02.680204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:02.680204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:02.680204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:02.680204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:02.680204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:02.680204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:02.680204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:02.680204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:02.680204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:02.681095 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:02.681095 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:02.681095 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:02.681095 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:02.681095 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:02.681095 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:02.681095 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:02.681095 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:02.681095 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:02.681095 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:02.681095 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:02.681095 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:02.681095 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:02.681095 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:02.681095 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:02.681095 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:02.682088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:02.682088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:02.682088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:02.682088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:02.682088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:02.682088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:02.682088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:02.682088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:02.682088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:02.682088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:02.682088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:07:02.682088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:02.682088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 78/166] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 31 00:07:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:02.682731 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633701a7-99be-41a6-9e4d-421bc6fd171d None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:02.744801 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:02.826732 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:02.849574 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 89/167] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:02.850886 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb76c8-1a1b-490b-beee-867e891cf3e8 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:02.926137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633701a7-99be-41a6-9e4d-421bc6fd171d None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:02.999753 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be2e119-700f-4a05-9996-89f4bb0914c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:02.999753 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2e119-700f-4a05-9996-89f4bb0914c4 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:03.001055 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2e119-700f-4a05-9996-89f4bb0914c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:03.001365 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2e119-700f-4a05-9996-89f4bb0914c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:03.031117 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be2e119-700f-4a05-9996-89f4bb0914c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:03.036677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-4be2e119-700f-4a05-9996-89f4bb0914c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:03.036677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:03.036677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:03.036677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:03.036677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:03.036677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:03.036677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:03.036677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:03.036677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:03.036677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:03.036677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:03.036677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:03.036677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:03.036677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:03.036677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:03.036677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:03.037544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:03.037544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:03.037544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:03.037544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:03.037544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:03.037544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:03.037544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:03.037544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:03.037544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:03.037544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:03.037544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:03.037544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:03.037544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:03.037544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:03.037544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:03.037544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:03.038328 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:03.038328 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:03.038328 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:03.038328 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:03.038328 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:03.038328 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:03.038328 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:03.038328 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:03.038328 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:03.038328 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:03.038328 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:03.038328 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:03.043399 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 79/168] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:03.043925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb76c8-1a1b-490b-beee-867e891cf3e8 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:03.044255 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be2e119-700f-4a05-9996-89f4bb0914c4 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:03.130546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91a793-f279-4940-8308-c58ab238905b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:03.131797 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91a793-f279-4940-8308-c58ab238905b None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:03.132263 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91a793-f279-4940-8308-c58ab238905b None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:03.132735 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91a793-f279-4940-8308-c58ab238905b None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:03.133587 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91a793-f279-4940-8308-c58ab238905b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:03.134067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91a793-f279-4940-8308-c58ab238905b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:03.135780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91a793-f279-4940-8308-c58ab238905b None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:03.148809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 90/169] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:07:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:03.149981 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91a793-f279-4940-8308-c58ab238905b None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:03.165426 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:03.191188 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be2e119-700f-4a05-9996-89f4bb0914c4 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:03.228425 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf58f81-db4c-41ba-aeed-24714e880db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:03.229308 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf58f81-db4c-41ba-aeed-24714e880db6 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:03.229720 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf58f81-db4c-41ba-aeed-24714e880db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:03.230150 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf58f81-db4c-41ba-aeed-24714e880db6 None admin] PATH_INFO: `/v3/projects/9eb64431e703423581c08ff8f8378eb3` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:03.231611 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf58f81-db4c-41ba-aeed-24714e880db6 None admin] RBAC: Authorizing `identity:get_project(project_id=9eb64431e703423581c08ff8f8378eb3)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:03.233808 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf58f81-db4c-41ba-aeed-24714e880db6 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:03.235803 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 80/170] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:03 2022] GET /identity/v3/projects/9eb64431e703423581c08ff8f8378eb3 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:03.236779 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf58f81-db4c-41ba-aeed-24714e880db6 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:03.256582 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:03.344409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91a793-f279-4940-8308-c58ab238905b None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:03.353025 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf58f81-db4c-41ba-aeed-24714e880db6 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:03.390096 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75062e6f-9d9a-4dfb-bddf-cf0ccf545b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:03.391018 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75062e6f-9d9a-4dfb-bddf-cf0ccf545b44 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:03.391533 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75062e6f-9d9a-4dfb-bddf-cf0ccf545b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:03.391995 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75062e6f-9d9a-4dfb-bddf-cf0ccf545b44 None admin] PATH_INFO: `/v3/users/8eb6244b69ca487281ed852a375261ca` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:03.393391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75062e6f-9d9a-4dfb-bddf-cf0ccf545b44 None admin] RBAC: Authorizing `identity:get_user(user_id=8eb6244b69ca487281ed852a375261ca)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:03.395114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75062e6f-9d9a-4dfb-bddf-cf0ccf545b44 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:03.396908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 81/171] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:07:03 2022] GET /identity/v3/users/8eb6244b69ca487281ed852a375261ca => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:03.398240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d04709-9d1f-437a-8223-02f5259ed0ba None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:03.403058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d04709-9d1f-437a-8223-02f5259ed0ba None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:03.403676 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d04709-9d1f-437a-8223-02f5259ed0ba None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:03.406313 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 82/172] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:03 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:03.406562 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a600da-f02c-49d0-b6d2-28856fa099aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:03.407280 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a600da-f02c-49d0-b6d2-28856fa099aa None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:03.407800 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a600da-f02c-49d0-b6d2-28856fa099aa None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:03.408090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a600da-f02c-49d0-b6d2-28856fa099aa None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:03.408433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d04709-9d1f-437a-8223-02f5259ed0ba None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:03.408770 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a600da-f02c-49d0-b6d2-28856fa099aa None admin] RBAC: Adding query filter params (name=member) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:03.409078 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a600da-f02c-49d0-b6d2-28856fa099aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:03.410466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a600da-f02c-49d0-b6d2-28856fa099aa None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:03.428515 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 91/173] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:07:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:03.436446 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a600da-f02c-49d0-b6d2-28856fa099aa None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:03.459405 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:03.522907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d04709-9d1f-437a-8223-02f5259ed0ba None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:03.577890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1aeb64-444f-4897-af96-eae05d83ba3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:03.578876 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1aeb64-444f-4897-af96-eae05d83ba3c None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:03.579383 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1aeb64-444f-4897-af96-eae05d83ba3c None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:03.579856 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1aeb64-444f-4897-af96-eae05d83ba3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:03.580881 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1aeb64-444f-4897-af96-eae05d83ba3c None admin] RBAC: Adding query filter params (role.id=cc8bd97c7b9745d1a52746650a2e49b9, scope.project.id=9eb64431e703423581c08ff8f8378eb3, user.id=8eb6244b69ca487281ed852a375261ca) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:03.581480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1aeb64-444f-4897-af96-eae05d83ba3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:03.583216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1aeb64-444f-4897-af96-eae05d83ba3c None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:03.590558 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 83/174] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:07:03 2022] GET /identity/v3/role_assignments?user.id=8eb6244b69ca487281ed852a375261ca&scope.project.id=9eb64431e703423581c08ff8f8378eb3&role.id=cc8bd97c7b9745d1a52746650a2e49b9 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:03.599916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8e0eb-d41f-45f6-b8d1-7d49b66fc326 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:03.600566 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8e0eb-d41f-45f6-b8d1-7d49b66fc326 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:03.601043 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8e0eb-d41f-45f6-b8d1-7d49b66fc326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:03.631217 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-89c8e0eb-d41f-45f6-b8d1-7d49b66fc326 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:03.632803 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-89c8e0eb-d41f-45f6-b8d1-7d49b66fc326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:03.670721 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a600da-f02c-49d0-b6d2-28856fa099aa None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:03.695486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:03.713492 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ec244-07b9-464e-93a6-5c30ad0ecba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:03.714245 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ec244-07b9-464e-93a6-5c30ad0ecba5 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:03.714533 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ec244-07b9-464e-93a6-5c30ad0ecba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:03.714997 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ec244-07b9-464e-93a6-5c30ad0ecba5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:03.817362 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 84/175] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:03.818029 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8e0eb-d41f-45f6-b8d1-7d49b66fc326 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:03.845383 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:03.862452 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ec244-07b9-464e-93a6-5c30ad0ecba5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:03.864234 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ec244-07b9-464e-93a6-5c30ad0ecba5 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:03.967612 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8e0eb-d41f-45f6-b8d1-7d49b66fc326 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:04.016685 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3231f0-7fdf-4cfc-b939-4157b9e90ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:04.017316 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 92/176] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:04.017533 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3231f0-7fdf-4cfc-b939-4157b9e90ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:04.017692 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3231f0-7fdf-4cfc-b939-4157b9e90ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:04.017927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3231f0-7fdf-4cfc-b939-4157b9e90ae1 None admin] PATH_INFO: `/v3/users/8eb6244b69ca487281ed852a375261ca` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:04.019395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3231f0-7fdf-4cfc-b939-4157b9e90ae1 None admin] RBAC: Authorizing `identity:get_user(user_id=8eb6244b69ca487281ed852a375261ca)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:04.019626 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9676d29-ebd5-4b13-9fe7-52a6c742c636 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:04.019916 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9676d29-ebd5-4b13-9fe7-52a6c742c636 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:04.020191 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9676d29-ebd5-4b13-9fe7-52a6c742c636 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:04.021453 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3231f0-7fdf-4cfc-b939-4157b9e90ae1 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:04.021575 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 93/177] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:04.022494 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005538f8-3f55-473a-9c53-dd56bcd7bb12 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:04.022776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005538f8-3f55-473a-9c53-dd56bcd7bb12 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:04.023062 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005538f8-3f55-473a-9c53-dd56bcd7bb12 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:04.024008 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 85/178] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:07:03 2022] GET /identity/v3/users/8eb6244b69ca487281ed852a375261ca => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:04.025232 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f1e95-47fb-4739-8589-234c369f4e32 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:04.025538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f1e95-47fb-4739-8589-234c369f4e32 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:04.026157 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f1e95-47fb-4739-8589-234c369f4e32 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:04.027175 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 94/179] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:04.027393 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 86/180] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:04.028257 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005538f8-3f55-473a-9c53-dd56bcd7bb12 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:04.028869 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a710f-34be-4a60-86b2-a49707926880 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:04.029334 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a710f-34be-4a60-86b2-a49707926880 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:04.029897 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a710f-34be-4a60-86b2-a49707926880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:04.044469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:04.054566 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-2a5a710f-34be-4a60-86b2-a49707926880 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:04.055781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-2a5a710f-34be-4a60-86b2-a49707926880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:04.128969 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 87/181] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:04.129862 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a710f-34be-4a60-86b2-a49707926880 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:04.144048 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:04.153707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005538f8-3f55-473a-9c53-dd56bcd7bb12 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:04.192872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9717eb-0c21-41a5-89b0-924beaf8676f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:04.193513 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9717eb-0c21-41a5-89b0-924beaf8676f None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:04.193791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9717eb-0c21-41a5-89b0-924beaf8676f None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:04.194077 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9717eb-0c21-41a5-89b0-924beaf8676f None admin] PATH_INFO: `/v3/projects/9eb64431e703423581c08ff8f8378eb3` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:04.195329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9717eb-0c21-41a5-89b0-924beaf8676f None admin] RBAC: Authorizing `identity:get_project(project_id=9eb64431e703423581c08ff8f8378eb3)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:04.196702 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9717eb-0c21-41a5-89b0-924beaf8676f None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:04.198336 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 95/182] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:04 2022] GET /identity/v3/projects/9eb64431e703423581c08ff8f8378eb3 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:04.204648 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a11cb1-d90c-4a26-9ba9-dffe53e1b368 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:04.205192 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a11cb1-d90c-4a26-9ba9-dffe53e1b368 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:04.205472 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a11cb1-d90c-4a26-9ba9-dffe53e1b368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:04.228406 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-32a11cb1-d90c-4a26-9ba9-dffe53e1b368 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:04.229808 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-32a11cb1-d90c-4a26-9ba9-dffe53e1b368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:04.241609 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a710f-34be-4a60-86b2-a49707926880 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:04.278535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284af3f0-834d-4ded-a6f4-c158a191b428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:04.279788 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284af3f0-834d-4ded-a6f4-c158a191b428 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:04.280221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284af3f0-834d-4ded-a6f4-c158a191b428 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:04.280624 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284af3f0-834d-4ded-a6f4-c158a191b428 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:04.304629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 96/183] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:04.306156 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11f1bf-abcd-4eea-a09c-6c9a944df0dd None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:04.306657 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11f1bf-abcd-4eea-a09c-6c9a944df0dd None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:04.307160 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11f1bf-abcd-4eea-a09c-6c9a944df0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:04.328992 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-7d11f1bf-abcd-4eea-a09c-6c9a944df0dd None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:04.330508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-7d11f1bf-abcd-4eea-a09c-6c9a944df0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:04.334973 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:04.393613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284af3f0-834d-4ded-a6f4-c158a191b428 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:04.396027 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284af3f0-834d-4ded-a6f4-c158a191b428 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:04.409955 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:04.410815 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 97/184] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:04.416037 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11f1bf-abcd-4eea-a09c-6c9a944df0dd None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:04.528818 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 88/185] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:04.531019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284af3f0-834d-4ded-a6f4-c158a191b428 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:04.536588 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11f1bf-abcd-4eea-a09c-6c9a944df0dd None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:04.573908 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe0343-fe84-4a31-801c-91710ac8a2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:04.574873 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe0343-fe84-4a31-801c-91710ac8a2bc None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:04.575384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe0343-fe84-4a31-801c-91710ac8a2bc None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:04.575881 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe0343-fe84-4a31-801c-91710ac8a2bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:04.584869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:04.588291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe0343-fe84-4a31-801c-91710ac8a2bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:04.595405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-0fbe0343-fe84-4a31-801c-91710ac8a2bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:07:04.595405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:04.595405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:04.595405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:04.595405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:04.595405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:04.595405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:04.595405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:04.595405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:04.595405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:04.595405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:04.595405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:04.595405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:04.595405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:04.595405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:04.595405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:04.596296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:04.596296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:04.596296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:04.596296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:04.596296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:04.596296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:04.596296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:04.596296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:04.596296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:04.596296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:04.596296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:04.596296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:04.596296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:04.596296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:04.596296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:04.596296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:04.597066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:04.597066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:04.597066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:04.597066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:04.597066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:04.597066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:04.597066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:04.597066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:04.597066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:04.597066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:04.597066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:07:04.597066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:04.599381 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 98/186] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:07:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:04.600373 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe0343-fe84-4a31-801c-91710ac8a2bc None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:04.650829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284af3f0-834d-4ded-a6f4-c158a191b428 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:04.660822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:04.688545 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705f739-8620-46b8-88c1-463fa55c26aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:04.689327 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705f739-8620-46b8-88c1-463fa55c26aa None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:04.689611 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705f739-8620-46b8-88c1-463fa55c26aa None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:04.689879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705f739-8620-46b8-88c1-463fa55c26aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:04.690771 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705f739-8620-46b8-88c1-463fa55c26aa None admin] RBAC: Adding query filter params (role.id=4d91c00f40304b9e9ef4e6c89f46e5c5, scope.project.id=9eb64431e703423581c08ff8f8378eb3, user.id=8eb6244b69ca487281ed852a375261ca) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:04.691164 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705f739-8620-46b8-88c1-463fa55c26aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:04.692759 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705f739-8620-46b8-88c1-463fa55c26aa None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:04.704559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 89/187] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:07:04 2022] GET /identity/v3/role_assignments?user.id=8eb6244b69ca487281ed852a375261ca&scope.project.id=9eb64431e703423581c08ff8f8378eb3&role.id=4d91c00f40304b9e9ef4e6c89f46e5c5 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:04.705419 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705f739-8620-46b8-88c1-463fa55c26aa None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:04.710917 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe0343-fe84-4a31-801c-91710ac8a2bc None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:04.751290 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a27a6-8671-4987-9593-54188f998e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:04.751955 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a27a6-8671-4987-9593-54188f998e04 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:04.752234 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a27a6-8671-4987-9593-54188f998e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:04.752505 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a27a6-8671-4987-9593-54188f998e04 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:04.770111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6a27a6-8671-4987-9593-54188f998e04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:04.775196 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-6e6a27a6-8671-4987-9593-54188f998e04 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:07:04.775196 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:04.775196 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:04.775196 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:04.775196 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:04.775196 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:04.775196 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:04.775196 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:04.775196 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:04.775196 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:04.775196 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:04.775196 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:04.775196 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:04.775196 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:04.775196 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:04.775196 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:04.776318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:04.776318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:04.776318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:04.776318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:04.776318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:04.776318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:04.776318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:04.776318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:04.776318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:04.776318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:04.776318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:04.776318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:04.776318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:04.776318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:04.776318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:04.776318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:04.777291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:04.777291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:04.777291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:04.777291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:04.777291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:04.777291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:04.777291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:04.777291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:04.777291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:04.777291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:04.777291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:07:04.777291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:04.777291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 99/188] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 31 00:07:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:04.777291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a27a6-8671-4987-9593-54188f998e04 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:04.836909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705f739-8620-46b8-88c1-463fa55c26aa None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:04.854273 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:04.879911 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16e54b-5f22-4b44-a643-a5ecf3ddf271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:04.880648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16e54b-5f22-4b44-a643-a5ecf3ddf271 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:04.880927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16e54b-5f22-4b44-a643-a5ecf3ddf271 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:04.881200 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16e54b-5f22-4b44-a643-a5ecf3ddf271 None admin] PATH_INFO: `/v3/projects/be6cea0351cb401184e319eef387800a` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:04.882337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c16e54b-5f22-4b44-a643-a5ecf3ddf271 None admin] RBAC: Authorizing `identity:get_project(project_id=be6cea0351cb401184e319eef387800a)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:04.883832 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c16e54b-5f22-4b44-a643-a5ecf3ddf271 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:04.885510 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 90/189] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:04 2022] GET /identity/v3/projects/be6cea0351cb401184e319eef387800a => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:04.886272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16e54b-5f22-4b44-a643-a5ecf3ddf271 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:04.906549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a27a6-8671-4987-9593-54188f998e04 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:04.937616 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:04.945399 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13379536-ecf5-464c-b3f3-297caeb3b78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:04.946036 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13379536-ecf5-464c-b3f3-297caeb3b78b None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:04.946318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13379536-ecf5-464c-b3f3-297caeb3b78b None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:04.946592 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13379536-ecf5-464c-b3f3-297caeb3b78b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:04.947714 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13379536-ecf5-464c-b3f3-297caeb3b78b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:04.949189 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13379536-ecf5-464c-b3f3-297caeb3b78b None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:04.950760 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 100/190] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:04.952321 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13379536-ecf5-464c-b3f3-297caeb3b78b None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:05.007848 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16e54b-5f22-4b44-a643-a5ecf3ddf271 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:05.041583 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181968eb-d223-4d61-892b-1286eeb25d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:05.042200 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181968eb-d223-4d61-892b-1286eeb25d8a None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:05.042479 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181968eb-d223-4d61-892b-1286eeb25d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:05.042748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181968eb-d223-4d61-892b-1286eeb25d8a None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:05.043456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181968eb-d223-4d61-892b-1286eeb25d8a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:05.043456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181968eb-d223-4d61-892b-1286eeb25d8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:05.045671 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181968eb-d223-4d61-892b-1286eeb25d8a None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:05.053766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 91/191] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:05.054084 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13379536-ecf5-464c-b3f3-297caeb3b78b None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:05.054561 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181968eb-d223-4d61-892b-1286eeb25d8a None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:05.080847 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:05.089388 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec97cf-28c6-4b6f-855f-be7c3f848dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:05.089956 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec97cf-28c6-4b6f-855f-be7c3f848dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:05.090232 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec97cf-28c6-4b6f-855f-be7c3f848dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:05.090509 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec97cf-28c6-4b6f-855f-be7c3f848dc7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:05.101236 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec97cf-28c6-4b6f-855f-be7c3f848dc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:05.106416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-c2ec97cf-28c6-4b6f-855f-be7c3f848dc7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:07:05.106416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:05.106416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:05.106416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:05.106416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:05.106416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:05.106416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:05.106416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:05.106416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:05.106416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:05.106416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:05.106416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:05.106416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:05.106416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:05.106416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:05.106416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:05.107541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:05.107541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:05.107541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:05.107541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:05.107541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:05.107541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:05.107541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:05.107541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:05.107541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:05.107541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:05.107541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:05.107541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:05.107541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:05.107541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:05.107541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:05.107541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:05.108352 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:05.108352 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:05.108352 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:05.108352 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:05.108352 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:05.108352 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:05.108352 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:05.108352 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:05.108352 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:05.108352 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:05.108352 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:07:05.108352 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:05.108352 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 101/192] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:05.108352 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec97cf-28c6-4b6f-855f-be7c3f848dc7 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:05.159840 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181968eb-d223-4d61-892b-1286eeb25d8a None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:05.161350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:05.194861 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1092bcd-ced9-4b69-be52-669704dc9cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:05.195492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1092bcd-ced9-4b69-be52-669704dc9cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:05.195774 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1092bcd-ced9-4b69-be52-669704dc9cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:05.196040 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1092bcd-ced9-4b69-be52-669704dc9cb5 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:05.196695 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1092bcd-ced9-4b69-be52-669704dc9cb5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:05.196991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1092bcd-ced9-4b69-be52-669704dc9cb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:05.198373 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1092bcd-ced9-4b69-be52-669704dc9cb5 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:05.206677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 92/193] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:07:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:05.206813 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec97cf-28c6-4b6f-855f-be7c3f848dc7 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:05.207471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1092bcd-ced9-4b69-be52-669704dc9cb5 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:05.240445 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6b4d6-ad83-44bb-a208-8f908ae8f358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:05.241045 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6b4d6-ad83-44bb-a208-8f908ae8f358 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:05.241319 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6b4d6-ad83-44bb-a208-8f908ae8f358 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:05.241587 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6b4d6-ad83-44bb-a208-8f908ae8f358 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:05.242308 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6b4d6-ad83-44bb-a208-8f908ae8f358 None admin] RBAC: Adding query filter params (role.id=cc8bd97c7b9745d1a52746650a2e49b9, scope.project.id=be6cea0351cb401184e319eef387800a, user.id=8eb6244b69ca487281ed852a375261ca) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:05.242613 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6b4d6-ad83-44bb-a208-8f908ae8f358 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:05.244083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6b4d6-ad83-44bb-a208-8f908ae8f358 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:05.249921 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 102/194] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:07:05 2022] GET /identity/v3/role_assignments?user.id=8eb6244b69ca487281ed852a375261ca&scope.project.id=be6cea0351cb401184e319eef387800a&role.id=cc8bd97c7b9745d1a52746650a2e49b9 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:05.252528 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6b4d6-ad83-44bb-a208-8f908ae8f358 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:05.314544 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:05.316612 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1092bcd-ced9-4b69-be52-669704dc9cb5 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:05.355986 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8946c-3828-4f5a-8c57-1ed86a93c3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:05.356527 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8946c-3828-4f5a-8c57-1ed86a93c3d5 None admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:05.356802 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8946c-3828-4f5a-8c57-1ed86a93c3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:05.357071 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8946c-3828-4f5a-8c57-1ed86a93c3d5 None admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:05.357946 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af8946c-3828-4f5a-8c57-1ed86a93c3d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:05.359603 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af8946c-3828-4f5a-8c57-1ed86a93c3d5 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:05.367473 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6b4d6-ad83-44bb-a208-8f908ae8f358 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:05.392447 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 93/195] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 31 00:07:05 2022] POST /identity/v3/users => generated 297 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:07:05.393394 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8946c-3828-4f5a-8c57-1ed86a93c3d5 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:05.415571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2edca8-cf79-41b9-b067-7e1029c4b867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:05.416342 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2edca8-cf79-41b9-b067-7e1029c4b867 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:05.416624 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2edca8-cf79-41b9-b067-7e1029c4b867 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:05.416897 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2edca8-cf79-41b9-b067-7e1029c4b867 None admin] PATH_INFO: `/v3/users/9c58d322d6f84ed5a67aafe90ebfb284` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:05.420532 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:05.438324 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2edca8-cf79-41b9-b067-7e1029c4b867 None admin] RBAC: Authorizing `identity:get_user(user_id=9c58d322d6f84ed5a67aafe90ebfb284)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:05.439852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2edca8-cf79-41b9-b067-7e1029c4b867 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:05.441544 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 103/196] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:07:05 2022] GET /identity/v3/users/9c58d322d6f84ed5a67aafe90ebfb284 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:05.442418 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2edca8-cf79-41b9-b067-7e1029c4b867 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:05.532519 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8946c-3828-4f5a-8c57-1ed86a93c3d5 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:05.580191 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ccba0a-6878-455b-b6e3-63f75aa7c5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:05.580810 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ccba0a-6878-455b-b6e3-63f75aa7c5c0 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:05.581085 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ccba0a-6878-455b-b6e3-63f75aa7c5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:05.581352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ccba0a-6878-455b-b6e3-63f75aa7c5c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:05.582022 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ccba0a-6878-455b-b6e3-63f75aa7c5c0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:05.582320 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ccba0a-6878-455b-b6e3-63f75aa7c5c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:05.583737 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ccba0a-6878-455b-b6e3-63f75aa7c5c0 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:05.584375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2edca8-cf79-41b9-b067-7e1029c4b867 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:05.593162 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 94/197] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:07:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:05.595134 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ccba0a-6878-455b-b6e3-63f75aa7c5c0 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:05.600899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:05.624760 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350023f7-4190-4d9a-8e86-810e23f660d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:05.625659 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350023f7-4190-4d9a-8e86-810e23f660d1 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:05.625874 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350023f7-4190-4d9a-8e86-810e23f660d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:05.626147 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350023f7-4190-4d9a-8e86-810e23f660d1 None admin] PATH_INFO: `/v3/users/9c58d322d6f84ed5a67aafe90ebfb284` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:05.627378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350023f7-4190-4d9a-8e86-810e23f660d1 None admin] RBAC: Authorizing `identity:get_user(user_id=9c58d322d6f84ed5a67aafe90ebfb284)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:05.628848 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350023f7-4190-4d9a-8e86-810e23f660d1 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:05.630593 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 104/198] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:07:05 2022] GET /identity/v3/users/9c58d322d6f84ed5a67aafe90ebfb284 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:05.631229 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350023f7-4190-4d9a-8e86-810e23f660d1 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:05.694126 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:05.722170 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ccba0a-6878-455b-b6e3-63f75aa7c5c0 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:05.766166 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02645f76-eaae-4506-b0fb-b393b6b26eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:05.766995 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02645f76-eaae-4506-b0fb-b393b6b26eec None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:05.767300 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02645f76-eaae-4506-b0fb-b393b6b26eec None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:05.767691 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02645f76-eaae-4506-b0fb-b393b6b26eec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:05.770586 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350023f7-4190-4d9a-8e86-810e23f660d1 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:05.809571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac38a12-ed2b-4cea-be8d-ab103c954820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:05.810273 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac38a12-ed2b-4cea-be8d-ab103c954820 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:05.811177 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac38a12-ed2b-4cea-be8d-ab103c954820 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:05.811448 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac38a12-ed2b-4cea-be8d-ab103c954820 None admin] PATH_INFO: `/v3/users/35871d61fe1e4c7186aa0cbe990ddba5` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:05.812610 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac38a12-ed2b-4cea-be8d-ab103c954820 None admin] RBAC: Authorizing `identity:get_user(user_id=35871d61fe1e4c7186aa0cbe990ddba5)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:05.814069 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac38a12-ed2b-4cea-be8d-ab103c954820 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:05.815824 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 105/199] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:07:05 2022] GET /identity/v3/users/35871d61fe1e4c7186aa0cbe990ddba5 => generated 289 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:05.816587 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac38a12-ed2b-4cea-be8d-ab103c954820 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:05.844214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02645f76-eaae-4506-b0fb-b393b6b26eec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:05.845731 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02645f76-eaae-4506-b0fb-b393b6b26eec None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:05.887684 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:05.935164 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:05.953598 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac38a12-ed2b-4cea-be8d-ab103c954820 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:05.958762 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 95/200] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:05.959884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02645f76-eaae-4506-b0fb-b393b6b26eec None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:06.001654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55512437-1359-4ab3-891e-67324cfa6916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:06.002539 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55512437-1359-4ab3-891e-67324cfa6916 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:06.002955 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55512437-1359-4ab3-891e-67324cfa6916 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:06.003383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55512437-1359-4ab3-891e-67324cfa6916 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:06.076467 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02645f76-eaae-4506-b0fb-b393b6b26eec None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:06.094409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55512437-1359-4ab3-891e-67324cfa6916 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:06.096029 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55512437-1359-4ab3-891e-67324cfa6916 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:06.134206 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2159e-b010-4ac0-bf06-ab43ffbf8ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:06.136385 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2159e-b010-4ac0-bf06-ab43ffbf8ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:06.136903 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2159e-b010-4ac0-bf06-ab43ffbf8ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:06.137378 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2159e-b010-4ac0-bf06-ab43ffbf8ed7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:06.149685 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:06.211084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:06.226161 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 106/201] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:06.226161 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55512437-1359-4ab3-891e-67324cfa6916 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:06.243015 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2159e-b010-4ac0-bf06-ab43ffbf8ed7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:06.244735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2159e-b010-4ac0-bf06-ab43ffbf8ed7 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:06.352866 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55512437-1359-4ab3-891e-67324cfa6916 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:06.373586 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 96/202] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:06.375752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2159e-b010-4ac0-bf06-ab43ffbf8ed7 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:06.390658 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89edfa55-be57-47ef-94e9-b9022c6d1cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:06.391310 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89edfa55-be57-47ef-94e9-b9022c6d1cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:06.391712 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89edfa55-be57-47ef-94e9-b9022c6d1cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:06.392087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89edfa55-be57-47ef-94e9-b9022c6d1cf1 None admin] PATH_INFO: `/v3/projects/a8b13074ade34e4d944896277be959f5` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:06.393784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89edfa55-be57-47ef-94e9-b9022c6d1cf1 None admin] RBAC: Authorizing `identity:get_project(project_id=a8b13074ade34e4d944896277be959f5)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:06.395534 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89edfa55-be57-47ef-94e9-b9022c6d1cf1 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:06.402712 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 107/203] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:06 2022] GET /identity/v3/projects/a8b13074ade34e4d944896277be959f5 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:06.402712 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89edfa55-be57-47ef-94e9-b9022c6d1cf1 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:06.424392 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:06.489435 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:06.508516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2159e-b010-4ac0-bf06-ab43ffbf8ed7 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:06.534128 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89edfa55-be57-47ef-94e9-b9022c6d1cf1 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:06.543889 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683eb7a-9a27-41bc-a39e-11273bddc522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:06.544530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683eb7a-9a27-41bc-a39e-11273bddc522 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:06.545390 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683eb7a-9a27-41bc-a39e-11273bddc522 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:06.545753 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683eb7a-9a27-41bc-a39e-11273bddc522 None admin] PATH_INFO: `/v3/projects/a8b13074ade34e4d944896277be959f5` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:06.546959 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683eb7a-9a27-41bc-a39e-11273bddc522 None admin] RBAC: Authorizing `identity:get_project(project_id=a8b13074ade34e4d944896277be959f5)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:06.548466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683eb7a-9a27-41bc-a39e-11273bddc522 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:06.550132 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 97/204] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:06 2022] GET /identity/v3/projects/a8b13074ade34e4d944896277be959f5 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:06.550902 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683eb7a-9a27-41bc-a39e-11273bddc522 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:06.577884 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3218e629-cbf1-4265-9c39-3beabc11514b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:06.578925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218e629-cbf1-4265-9c39-3beabc11514b None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:06.579451 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218e629-cbf1-4265-9c39-3beabc11514b None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:06.579938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218e629-cbf1-4265-9c39-3beabc11514b None admin] PATH_INFO: `/v3/projects/9eb64431e703423581c08ff8f8378eb3` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:06.581333 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3218e629-cbf1-4265-9c39-3beabc11514b None admin] RBAC: Authorizing `identity:get_project(project_id=9eb64431e703423581c08ff8f8378eb3)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:06.582995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3218e629-cbf1-4265-9c39-3beabc11514b None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:06.584899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 108/205] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:06 2022] GET /identity/v3/projects/9eb64431e703423581c08ff8f8378eb3 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:06.586018 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3218e629-cbf1-4265-9c39-3beabc11514b None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:06.656628 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683eb7a-9a27-41bc-a39e-11273bddc522 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:06.687914 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:06.692644 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b9ad1-1088-4e01-87e8-51590682e5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:06.693270 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b9ad1-1088-4e01-87e8-51590682e5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:06.693554 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b9ad1-1088-4e01-87e8-51590682e5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:06.693882 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b9ad1-1088-4e01-87e8-51590682e5b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:06.694576 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2b9ad1-1088-4e01-87e8-51590682e5b6 None admin] RBAC: Adding query filter params (role.id=34df06ce84ad4cf8b3bd1a4490228017, scope.project.id=a8b13074ade34e4d944896277be959f5, user.id=9c58d322d6f84ed5a67aafe90ebfb284) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:06.694889 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2b9ad1-1088-4e01-87e8-51590682e5b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:06.696449 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2b9ad1-1088-4e01-87e8-51590682e5b6 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:06.711072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 98/206] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:07:06 2022] GET /identity/v3/role_assignments?user.id=9c58d322d6f84ed5a67aafe90ebfb284&scope.project.id=a8b13074ade34e4d944896277be959f5&role.id=34df06ce84ad4cf8b3bd1a4490228017 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:06.711072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b9ad1-1088-4e01-87e8-51590682e5b6 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:06.731562 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:06.749386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3218e629-cbf1-4265-9c39-3beabc11514b None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:06.825491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66938f08-566b-4f54-9fa8-565937c23d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:06.828159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66938f08-566b-4f54-9fa8-565937c23d99 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:06.828603 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66938f08-566b-4f54-9fa8-565937c23d99 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:06.829368 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66938f08-566b-4f54-9fa8-565937c23d99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:06.831446 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66938f08-566b-4f54-9fa8-565937c23d99 None admin] RBAC: Adding query filter params (role.id=4d91c00f40304b9e9ef4e6c89f46e5c5, scope.project.id=a8b13074ade34e4d944896277be959f5, user.id=9c58d322d6f84ed5a67aafe90ebfb284) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:06.831941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66938f08-566b-4f54-9fa8-565937c23d99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:06.835273 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66938f08-566b-4f54-9fa8-565937c23d99 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:06.846297 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 109/207] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:07:06 2022] GET /identity/v3/role_assignments?user.id=9c58d322d6f84ed5a67aafe90ebfb284&scope.project.id=a8b13074ade34e4d944896277be959f5&role.id=4d91c00f40304b9e9ef4e6c89f46e5c5 => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:06.880544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b9ad1-1088-4e01-87e8-51590682e5b6 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:06.920423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc4dca-ce04-4798-a0ba-7cada931f6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:06.920981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc4dca-ce04-4798-a0ba-7cada931f6af None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:06.921268 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc4dca-ce04-4798-a0ba-7cada931f6af None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:06.922539 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc4dca-ce04-4798-a0ba-7cada931f6af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:06.922539 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc4dca-ce04-4798-a0ba-7cada931f6af None admin] RBAC: Adding query filter params (role.id=65d36a76676c424db12c090f01595bdc, scope.project.id=9eb64431e703423581c08ff8f8378eb3, user.id=35871d61fe1e4c7186aa0cbe990ddba5) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:06.922709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc4dca-ce04-4798-a0ba-7cada931f6af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:06.924210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc4dca-ce04-4798-a0ba-7cada931f6af None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:06.933603 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 99/208] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:07:06 2022] GET /identity/v3/role_assignments?user.id=35871d61fe1e4c7186aa0cbe990ddba5&scope.project.id=9eb64431e703423581c08ff8f8378eb3&role.id=65d36a76676c424db12c090f01595bdc => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:07.787494 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4f3ec-3b5f-4dd4-a2b6-ef2433cba187 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:07.788451 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4f3ec-3b5f-4dd4-a2b6-ef2433cba187 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:07.788953 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4f3ec-3b5f-4dd4-a2b6-ef2433cba187 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:07.790753 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 110/209] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:07.796462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7247b18-c466-4f8f-8e57-6f25ffe02321 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:07.796753 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7247b18-c466-4f8f-8e57-6f25ffe02321 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:07.797034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7247b18-c466-4f8f-8e57-6f25ffe02321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:07.816232 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-b7247b18-c466-4f8f-8e57-6f25ffe02321 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:07.817425 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b7247b18-c466-4f8f-8e57-6f25ffe02321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:07.825585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b63cf-3671-4fc5-ae07-ae85f6ac0d56 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:07.825933 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b63cf-3671-4fc5-ae07-ae85f6ac0d56 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:07.826833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b63cf-3671-4fc5-ae07-ae85f6ac0d56 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:07.828468 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 111/210] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:07:07.834006 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c25b7-ab6d-49a7-b592-2386f8ebdcdc None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:07.834541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c25b7-ab6d-49a7-b592-2386f8ebdcdc None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:07.834888 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c25b7-ab6d-49a7-b592-2386f8ebdcdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:07.854867 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-000c25b7-ab6d-49a7-b592-2386f8ebdcdc None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:07.856175 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-000c25b7-ab6d-49a7-b592-2386f8ebdcdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:07.893620 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 100/211] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:07.934265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 112/212] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:08.174639 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7247b18-c466-4f8f-8e57-6f25ffe02321 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:08.185083 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:08.224489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c25b7-ab6d-49a7-b592-2386f8ebdcdc None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:08.282925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7247b18-c466-4f8f-8e57-6f25ffe02321 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:08.318863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e62892-a8c7-4556-aabc-b18660f8c147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:08.319539 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e62892-a8c7-4556-aabc-b18660f8c147 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:08.319676 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:08.319819 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e62892-a8c7-4556-aabc-b18660f8c147 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:08.320109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e62892-a8c7-4556-aabc-b18660f8c147 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:08.333304 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e62892-a8c7-4556-aabc-b18660f8c147 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:08.338109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-33e62892-a8c7-4556-aabc-b18660f8c147 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:08.338109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:08.338109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:08.338109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:08.338109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:08.338109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:08.338109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:08.338109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:08.338109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:08.338109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:08.338109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:08.338109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:08.338109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:08.338109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:08.338109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:08.338109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:08.339222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:08.339222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:08.339222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:08.339222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:08.339222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:08.339222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:08.339222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:08.339222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:08.339222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:08.339222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:08.339222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:08.339222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:08.339222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:08.339222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:08.339222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:08.339222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:08.340236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:08.340236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:08.340236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:08.340236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:08.340236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:08.340236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:08.340236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:08.340236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:08.340236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:08.340236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:08.340236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:08.340236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:08.344461 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 101/213] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:08.345458 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e62892-a8c7-4556-aabc-b18660f8c147 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:08.346456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c25b7-ab6d-49a7-b592-2386f8ebdcdc None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:08.385853 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed55229-de7c-4cda-8878-52ea460740b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:08.386737 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed55229-de7c-4cda-8878-52ea460740b8 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:08.387211 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed55229-de7c-4cda-8878-52ea460740b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:08.387761 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed55229-de7c-4cda-8878-52ea460740b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:08.390310 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed55229-de7c-4cda-8878-52ea460740b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:08.392611 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed55229-de7c-4cda-8878-52ea460740b8 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:08.394398 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 113/214] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:08.400142 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed55229-de7c-4cda-8878-52ea460740b8 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:08.415610 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:08.450717 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e62892-a8c7-4556-aabc-b18660f8c147 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:08.486645 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47320ec-7bfa-49fd-91d2-c6c7e6bb0180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:08.487303 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47320ec-7bfa-49fd-91d2-c6c7e6bb0180 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:08.487575 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47320ec-7bfa-49fd-91d2-c6c7e6bb0180 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:08.487847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47320ec-7bfa-49fd-91d2-c6c7e6bb0180 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:08.488518 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47320ec-7bfa-49fd-91d2-c6c7e6bb0180 None admin] RBAC: Adding query filter params (name=member) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:08.488823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47320ec-7bfa-49fd-91d2-c6c7e6bb0180 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:08.490511 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47320ec-7bfa-49fd-91d2-c6c7e6bb0180 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:08.504498 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 102/215] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:07:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:08.505559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47320ec-7bfa-49fd-91d2-c6c7e6bb0180 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:08.521806 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed55229-de7c-4cda-8878-52ea460740b8 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:08.562423 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae11463-f269-447b-8925-81083907a742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:08.563312 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae11463-f269-447b-8925-81083907a742 None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:08.563839 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae11463-f269-447b-8925-81083907a742 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:08.564281 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae11463-f269-447b-8925-81083907a742 None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:08.565198 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae11463-f269-447b-8925-81083907a742 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:08.566826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae11463-f269-447b-8925-81083907a742 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:08.587578 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:08.605012 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 114/216] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 31 00:07:08 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:07:08.612837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47320ec-7bfa-49fd-91d2-c6c7e6bb0180 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:08.646698 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:08.652720 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc7402-94bd-41a1-8620-69b4d4fb94d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:08.653424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc7402-94bd-41a1-8620-69b4d4fb94d6 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:08.654344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc7402-94bd-41a1-8620-69b4d4fb94d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:08.655002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc7402-94bd-41a1-8620-69b4d4fb94d6 None admin] PATH_INFO: `/v3/users/53d04e52fcba4793872c2660f8550a4d` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:08.677495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc7402-94bd-41a1-8620-69b4d4fb94d6 None admin] RBAC: Authorizing `identity:get_user(user_id=53d04e52fcba4793872c2660f8550a4d)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:08.680951 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc7402-94bd-41a1-8620-69b4d4fb94d6 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:08.683905 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 103/217] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:07:08 2022] GET /identity/v3/users/53d04e52fcba4793872c2660f8550a4d => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:08.692228 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae11463-f269-447b-8925-81083907a742 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:08.817513 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae11463-f269-447b-8925-81083907a742 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:08.854538 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf84f8-c881-4a41-aaa9-6afceb5654da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:08.855239 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf84f8-c881-4a41-aaa9-6afceb5654da None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:08.855521 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf84f8-c881-4a41-aaa9-6afceb5654da None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:08.855853 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf84f8-c881-4a41-aaa9-6afceb5654da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:08.898096 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55361a3-4886-4e45-afe1-8111b36fd8ce None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:08.898615 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55361a3-4886-4e45-afe1-8111b36fd8ce None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:08.898615 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55361a3-4886-4e45-afe1-8111b36fd8ce None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:08.900007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 104/218] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:08.907225 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f3dec-9505-491e-9e84-4d98b6603487 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:08.907511 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f3dec-9505-491e-9e84-4d98b6603487 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:08.907787 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f3dec-9505-491e-9e84-4d98b6603487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:08.937774 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-865f3dec-9505-491e-9e84-4d98b6603487 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:08.939193 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:08.939570 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-865f3dec-9505-491e-9e84-4d98b6603487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:08.940714 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf84f8-c881-4a41-aaa9-6afceb5654da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:08.942495 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf84f8-c881-4a41-aaa9-6afceb5654da None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:09.043902 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 105/219] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:09.077976 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 115/220] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:09.083939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f3dec-9505-491e-9e84-4d98b6603487 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:09.120897 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749c88c-2678-4e5f-80d7-1ee84652559a None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:09.121300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749c88c-2678-4e5f-80d7-1ee84652559a None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:09.121671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749c88c-2678-4e5f-80d7-1ee84652559a None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:09.123967 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 116/221] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:09.130697 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c7803-a411-4d99-a7fd-19d11c766a67 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:09.131259 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c7803-a411-4d99-a7fd-19d11c766a67 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:09.131635 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c7803-a411-4d99-a7fd-19d11c766a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:09.174302 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-137c7803-a411-4d99-a7fd-19d11c766a67 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:09.175769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-137c7803-a411-4d99-a7fd-19d11c766a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:09.199607 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f3dec-9505-491e-9e84-4d98b6603487 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:09.226886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:09.237818 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0e455-3bab-409b-b38e-cd24adfcf634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:09.238856 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0e455-3bab-409b-b38e-cd24adfcf634 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:09.239349 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0e455-3bab-409b-b38e-cd24adfcf634 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:09.239812 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0e455-3bab-409b-b38e-cd24adfcf634 None admin] PATH_INFO: `/v3/projects/a8b13074ade34e4d944896277be959f5` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:09.241141 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0e455-3bab-409b-b38e-cd24adfcf634 None admin] RBAC: Authorizing `identity:get_project(project_id=a8b13074ade34e4d944896277be959f5)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:09.242815 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0e455-3bab-409b-b38e-cd24adfcf634 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:09.244682 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 106/222] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:09 2022] GET /identity/v3/projects/a8b13074ade34e4d944896277be959f5 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:09.251863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0e455-3bab-409b-b38e-cd24adfcf634 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:09.348735 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 117/223] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:09.373488 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0e455-3bab-409b-b38e-cd24adfcf634 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:09.409805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf3879-1c11-4fc1-8300-0a345aae6f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:09.410427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf3879-1c11-4fc1-8300-0a345aae6f7d None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:09.410744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf3879-1c11-4fc1-8300-0a345aae6f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:09.411031 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf3879-1c11-4fc1-8300-0a345aae6f7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:09.411859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf3879-1c11-4fc1-8300-0a345aae6f7d None admin] RBAC: Adding query filter params (role.id=cc8bd97c7b9745d1a52746650a2e49b9, scope.project.id=a8b13074ade34e4d944896277be959f5, user.id=53d04e52fcba4793872c2660f8550a4d) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:09.412167 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf3879-1c11-4fc1-8300-0a345aae6f7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:09.414080 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf3879-1c11-4fc1-8300-0a345aae6f7d None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:09.415818 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c7803-a411-4d99-a7fd-19d11c766a67 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:09.421769 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 107/224] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:07:09 2022] GET /identity/v3/role_assignments?user.id=53d04e52fcba4793872c2660f8550a4d&scope.project.id=a8b13074ade34e4d944896277be959f5&role.id=cc8bd97c7b9745d1a52746650a2e49b9 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:09.453761 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:09.551442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c7803-a411-4d99-a7fd-19d11c766a67 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:09.604046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55041187-c2eb-4653-b131-b857fa3adcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:09.604607 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55041187-c2eb-4653-b131-b857fa3adcd2 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:09.604889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55041187-c2eb-4653-b131-b857fa3adcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:09.605164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55041187-c2eb-4653-b131-b857fa3adcd2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:09.622469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55041187-c2eb-4653-b131-b857fa3adcd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:09.627952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-55041187-c2eb-4653-b131-b857fa3adcd2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:07:09.627952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:09.627952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:09.627952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:09.627952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:09.627952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:09.627952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:09.627952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:09.627952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:09.627952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:09.627952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:09.627952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:09.627952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:09.627952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:09.627952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:09.627952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:09.628809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:09.628809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:09.628809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:09.628809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:09.628809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:09.628809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:09.628809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:09.628809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:09.628809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:09.628809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:09.628809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:09.628809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:09.628809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:09.628809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:09.628809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:09.628809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:09.629593 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:09.629593 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:09.629593 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:09.629593 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:09.629593 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:09.629593 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:09.629593 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:09.629593 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:09.629593 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:09.629593 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:09.629593 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:07:09.629593 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:09.629593 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 118/225] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:07:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:07:09.635451 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf3879-1c11-4fc1-8300-0a345aae6f7d None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:09.656408 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55041187-c2eb-4653-b131-b857fa3adcd2 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:09.711333 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:09.783684 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:09.788769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55041187-c2eb-4653-b131-b857fa3adcd2 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:09.816609 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf3879-1c11-4fc1-8300-0a345aae6f7d None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:09.832468 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d74772-82f7-4850-b9d0-4246cdf19622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:09.833124 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d74772-82f7-4850-b9d0-4246cdf19622 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:09.833406 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d74772-82f7-4850-b9d0-4246cdf19622 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:09.833684 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d74772-82f7-4850-b9d0-4246cdf19622 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:09.845753 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d74772-82f7-4850-b9d0-4246cdf19622 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:09.850726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-13d74772-82f7-4850-b9d0-4246cdf19622 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:07:09.850726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:09.850726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:09.850726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:09.850726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:09.850726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:09.850726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:09.850726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:09.850726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:09.850726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:09.850726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:09.850726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:09.850726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:09.850726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:09.850726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:09.850726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:09.851655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:09.851655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:09.851655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:09.851655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:09.851655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:09.851655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:09.851655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:09.851655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:09.851655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:09.851655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:09.851655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:09.851655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:09.851655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:09.851655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:09.851655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:09.851655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:09.852453 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:09.852453 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:09.852453 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:09.852453 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:09.852453 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:09.852453 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:09.852453 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:09.852453 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:09.852453 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:09.852453 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:09.852453 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:07:09.852453 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:09.852453 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 119/226] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 31 00:07:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:09.858391 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d74772-82f7-4850-b9d0-4246cdf19622 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:09.859599 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f929b33-d961-4eaa-8915-f691e586557f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:09.860353 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f929b33-d961-4eaa-8915-f691e586557f None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:09.860795 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f929b33-d961-4eaa-8915-f691e586557f None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:09.861178 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f929b33-d961-4eaa-8915-f691e586557f None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:09.862088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f929b33-d961-4eaa-8915-f691e586557f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:09.862396 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f929b33-d961-4eaa-8915-f691e586557f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:09.864750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f929b33-d961-4eaa-8915-f691e586557f None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:09.880042 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 108/227] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:09.889110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f929b33-d961-4eaa-8915-f691e586557f None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:09.994006 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d74772-82f7-4850-b9d0-4246cdf19622 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:10.056662 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1aa774-fcd1-42af-bf16-027ac96e933b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:10.057284 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1aa774-fcd1-42af-bf16-027ac96e933b None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:10.057567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1aa774-fcd1-42af-bf16-027ac96e933b None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:10.057844 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1aa774-fcd1-42af-bf16-027ac96e933b None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:10.058521 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1aa774-fcd1-42af-bf16-027ac96e933b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:10.058829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1aa774-fcd1-42af-bf16-027ac96e933b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:10.061249 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1aa774-fcd1-42af-bf16-027ac96e933b None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:10.070209 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 120/228] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:07:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:10.073101 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f929b33-d961-4eaa-8915-f691e586557f None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:10.077377 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1aa774-fcd1-42af-bf16-027ac96e933b None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:10.115547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:10.123685 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:10.154774 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a3c4f-63ce-4cc2-9534-ee185ca6dee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:10.155617 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a3c4f-63ce-4cc2-9534-ee185ca6dee0 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:10.156003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a3c4f-63ce-4cc2-9534-ee185ca6dee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:10.156668 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a3c4f-63ce-4cc2-9534-ee185ca6dee0 None admin] PATH_INFO: `/v3/users/9c58d322d6f84ed5a67aafe90ebfb284` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:10.158210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a3c4f-63ce-4cc2-9534-ee185ca6dee0 None admin] RBAC: Authorizing `identity:get_user(user_id=9c58d322d6f84ed5a67aafe90ebfb284)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:10.159964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a3c4f-63ce-4cc2-9534-ee185ca6dee0 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:10.162849 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 109/229] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:07:09 2022] GET /identity/v3/users/9c58d322d6f84ed5a67aafe90ebfb284 => generated 283 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:10.170679 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a3c4f-63ce-4cc2-9534-ee185ca6dee0 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:10.314206 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1aa774-fcd1-42af-bf16-027ac96e933b None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:10.347528 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a3c4f-63ce-4cc2-9534-ee185ca6dee0 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:10.376021 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445b9dc-915a-4519-95f9-34870285a3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:10.376583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445b9dc-915a-4519-95f9-34870285a3fa None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:10.376857 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445b9dc-915a-4519-95f9-34870285a3fa None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:10.377128 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445b9dc-915a-4519-95f9-34870285a3fa None admin] PATH_INFO: `/v3/users/9c58d322d6f84ed5a67aafe90ebfb284` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:10.378500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445b9dc-915a-4519-95f9-34870285a3fa None admin] RBAC: Authorizing `identity:get_user(user_id=9c58d322d6f84ed5a67aafe90ebfb284)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:10.380009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445b9dc-915a-4519-95f9-34870285a3fa None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:10.382929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 121/230] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:07:10 2022] GET /identity/v3/users/9c58d322d6f84ed5a67aafe90ebfb284 => generated 283 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:10.388942 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445b9dc-915a-4519-95f9-34870285a3fa None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:10.431759 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96085740-b276-4b68-b7a6-071046906c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:10.432097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96085740-b276-4b68-b7a6-071046906c20 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:10.432285 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96085740-b276-4b68-b7a6-071046906c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:10.432558 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96085740-b276-4b68-b7a6-071046906c20 None admin] PATH_INFO: `/v3/projects/a8b13074ade34e4d944896277be959f5` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:10.433680 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96085740-b276-4b68-b7a6-071046906c20 None admin] RBAC: Authorizing `identity:get_project(project_id=a8b13074ade34e4d944896277be959f5)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:10.443207 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96085740-b276-4b68-b7a6-071046906c20 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:10.445214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 110/231] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:10 2022] GET /identity/v3/projects/a8b13074ade34e4d944896277be959f5 => generated 302 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:10.450642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96085740-b276-4b68-b7a6-071046906c20 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:10.486647 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:10.511428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445b9dc-915a-4519-95f9-34870285a3fa None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:10.556916 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29f35d-d479-4a8b-977d-7c64ed6a9169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:10.557911 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29f35d-d479-4a8b-977d-7c64ed6a9169 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:10.558425 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29f35d-d479-4a8b-977d-7c64ed6a9169 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:10.558891 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29f35d-d479-4a8b-977d-7c64ed6a9169 None admin] PATH_INFO: `/v3/projects/a8b13074ade34e4d944896277be959f5` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:10.562283 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29f35d-d479-4a8b-977d-7c64ed6a9169 None admin] RBAC: Authorizing `identity:get_project(project_id=a8b13074ade34e4d944896277be959f5)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:10.562405 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:10.564100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29f35d-d479-4a8b-977d-7c64ed6a9169 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:10.567171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 122/232] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:10 2022] GET /identity/v3/projects/a8b13074ade34e4d944896277be959f5 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:10.578732 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29f35d-d479-4a8b-977d-7c64ed6a9169 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:10.633809 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96085740-b276-4b68-b7a6-071046906c20 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:10.674670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf4501-4ae2-427a-885b-75f0b432928a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:10.676171 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf4501-4ae2-427a-885b-75f0b432928a None admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:10.676655 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf4501-4ae2-427a-885b-75f0b432928a None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:10.677220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf4501-4ae2-427a-885b-75f0b432928a None admin] PATH_INFO: `/v3/projects/a8b13074ade34e4d944896277be959f5/users/9c58d322d6f84ed5a67aafe90ebfb284/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:10.679942 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf4501-4ae2-427a-885b-75f0b432928a None admin] RBAC: Authorizing `identity:create_grant(project_id=a8b13074ade34e4d944896277be959f5, user_id=9c58d322d6f84ed5a67aafe90ebfb284, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:10.681756 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf4501-4ae2-427a-885b-75f0b432928a None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:10.697798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 111/233] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 31 00:07:10 2022] PUT /identity/v3/projects/a8b13074ade34e4d944896277be959f5/users/9c58d322d6f84ed5a67aafe90ebfb284/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:07:10.752453 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29f35d-d479-4a8b-977d-7c64ed6a9169 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:10.792382 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db52a58-5cbe-41a9-9a7e-3da7387fcd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:10.793429 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db52a58-5cbe-41a9-9a7e-3da7387fcd73 None admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:10.793934 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db52a58-5cbe-41a9-9a7e-3da7387fcd73 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:10.794434 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db52a58-5cbe-41a9-9a7e-3da7387fcd73 None admin] PATH_INFO: `/v3/projects/a8b13074ade34e4d944896277be959f5/users/9c58d322d6f84ed5a67aafe90ebfb284/roles/4d91c00f40304b9e9ef4e6c89f46e5c5` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:10.797786 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db52a58-5cbe-41a9-9a7e-3da7387fcd73 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8b13074ade34e4d944896277be959f5, user_id=9c58d322d6f84ed5a67aafe90ebfb284, role_id=4d91c00f40304b9e9ef4e6c89f46e5c5)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:10.799600 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db52a58-5cbe-41a9-9a7e-3da7387fcd73 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:10.816877 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 123/234] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 31 00:07:10 2022] PUT /identity/v3/projects/a8b13074ade34e4d944896277be959f5/users/9c58d322d6f84ed5a67aafe90ebfb284/roles/4d91c00f40304b9e9ef4e6c89f46e5c5 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:07:11.113571 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f4b9f-d61a-45ba-9659-4ae35b47cfad None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:11.114088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f4b9f-d61a-45ba-9659-4ae35b47cfad None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:11.114172 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f4b9f-d61a-45ba-9659-4ae35b47cfad None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:11.115756 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 112/235] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:11.120663 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b925c8-8e6c-4410-aae0-fb07c909b31d None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:11.121100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b925c8-8e6c-4410-aae0-fb07c909b31d None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:11.121279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b925c8-8e6c-4410-aae0-fb07c909b31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:11.131905 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f5d0b-8d2e-4f2f-82ff-d96668dddbd5 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:11.132208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f5d0b-8d2e-4f2f-82ff-d96668dddbd5 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:11.132469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f5d0b-8d2e-4f2f-82ff-d96668dddbd5 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:11.133814 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 113/236] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:07:11.145033 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-69b925c8-8e6c-4410-aae0-fb07c909b31d None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:11.146385 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-69b925c8-8e6c-4410-aae0-fb07c909b31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:11.148894 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:11.152180 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c8281-593e-4aee-a54b-2f477724c979 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:11.152464 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c8281-593e-4aee-a54b-2f477724c979 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:11.152741 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c8281-593e-4aee-a54b-2f477724c979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:11.178990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-9c9c8281-593e-4aee-a54b-2f477724c979 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:11.180575 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-9c9c8281-593e-4aee-a54b-2f477724c979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:11.259225 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 114/237] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:11.283370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 124/238] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:11.613898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c8281-593e-4aee-a54b-2f477724c979 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:11.661195 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:11.699437 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b925c8-8e6c-4410-aae0-fb07c909b31d None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:11.770997 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c8281-593e-4aee-a54b-2f477724c979 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:11.840109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18dba4-0a2c-4803-ac08-66da66e51851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:11.840765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18dba4-0a2c-4803-ac08-66da66e51851 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:11.842032 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18dba4-0a2c-4803-ac08-66da66e51851 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:11.842312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18dba4-0a2c-4803-ac08-66da66e51851 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:11.864379 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18dba4-0a2c-4803-ac08-66da66e51851 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:11.871559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-8c18dba4-0a2c-4803-ac08-66da66e51851 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:07:11.871559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:11.871559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:11.871559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:11.871559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:11.871559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:11.871559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:11.871559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:11.871559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:11.871559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:11.871559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:11.871559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:11.871559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:11.871559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:11.871559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:11.871559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:11.872417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:11.872417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:11.872417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:11.872417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:11.872417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:11.872417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:11.872417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:11.872417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:11.872417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:11.872417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:11.872417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:11.872417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:11.872417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:11.872417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:11.872417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:11.872417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:11.873739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:11.873739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:11.873739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:11.873739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:11.873739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:11.873739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:11.873739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:11.873739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:11.873739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:11.873739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:11.873739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:07:11.873739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:11.877761 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 115/239] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:07:11.881278 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18dba4-0a2c-4803-ac08-66da66e51851 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:11.901434 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b925c8-8e6c-4410-aae0-fb07c909b31d None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:11.965376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21eade-f7d8-4b94-bc6a-ba7c4d1f8495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:11.966826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21eade-f7d8-4b94-bc6a-ba7c4d1f8495 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:11.967461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21eade-f7d8-4b94-bc6a-ba7c4d1f8495 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:11.967908 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21eade-f7d8-4b94-bc6a-ba7c4d1f8495 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:11.969213 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21eade-f7d8-4b94-bc6a-ba7c4d1f8495 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:11.970831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21eade-f7d8-4b94-bc6a-ba7c4d1f8495 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:11.972601 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 125/240] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:11.995310 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21eade-f7d8-4b94-bc6a-ba7c4d1f8495 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:12.015379 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:12.092741 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18dba4-0a2c-4803-ac08-66da66e51851 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:12.095510 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:12.173627 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54069e-d3fe-43af-b3de-1ddb3a981af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:12.174233 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54069e-d3fe-43af-b3de-1ddb3a981af4 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:12.174469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54069e-d3fe-43af-b3de-1ddb3a981af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:12.174704 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54069e-d3fe-43af-b3de-1ddb3a981af4 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:12.175404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54069e-d3fe-43af-b3de-1ddb3a981af4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:12.175717 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54069e-d3fe-43af-b3de-1ddb3a981af4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:12.177102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54069e-d3fe-43af-b3de-1ddb3a981af4 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:12.198368 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 116/241] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:07:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:12.205109 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21eade-f7d8-4b94-bc6a-ba7c4d1f8495 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:12.207498 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54069e-d3fe-43af-b3de-1ddb3a981af4 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:12.260164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21a01e-c56c-499b-865c-0be4b3443dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:12.261046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21a01e-c56c-499b-865c-0be4b3443dcc None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:12.261336 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21a01e-c56c-499b-865c-0be4b3443dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:12.261614 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21a01e-c56c-499b-865c-0be4b3443dcc None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:12.262488 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21a01e-c56c-499b-865c-0be4b3443dcc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:12.265406 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21a01e-c56c-499b-865c-0be4b3443dcc None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:12.295319 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 126/242] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 31 00:07:11 2022] POST /identity/v3/users => generated 293 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:07:12.297284 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf06b4b-3410-4fc4-8788-83a6d126914c None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:12.297580 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf06b4b-3410-4fc4-8788-83a6d126914c None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:12.297859 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf06b4b-3410-4fc4-8788-83a6d126914c None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:12.299982 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 127/243] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:12.305673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9995099-3630-425d-9a0c-6a3da0ead000 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:12.306005 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9995099-3630-425d-9a0c-6a3da0ead000 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:12.306973 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9995099-3630-425d-9a0c-6a3da0ead000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:12.332140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-e9995099-3630-425d-9a0c-6a3da0ead000 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:12.334074 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e9995099-3630-425d-9a0c-6a3da0ead000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:12.334900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54069e-d3fe-43af-b3de-1ddb3a981af4 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:12.382051 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700feed-4b02-48c2-ad43-d0d14e02d7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:12.382761 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700feed-4b02-48c2-ad43-d0d14e02d7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:12.383067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700feed-4b02-48c2-ad43-d0d14e02d7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:12.383366 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700feed-4b02-48c2-ad43-d0d14e02d7e0 None admin] PATH_INFO: `/v3/users/c9fcdd29f7d741a78e3fc08852ad17a3` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:12.403583 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6700feed-4b02-48c2-ad43-d0d14e02d7e0 None admin] RBAC: Authorizing `identity:get_user(user_id=c9fcdd29f7d741a78e3fc08852ad17a3)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:12.405846 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6700feed-4b02-48c2-ad43-d0d14e02d7e0 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:12.407761 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 117/244] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:07:12 2022] GET /identity/v3/users/c9fcdd29f7d741a78e3fc08852ad17a3 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:12.415646 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700feed-4b02-48c2-ad43-d0d14e02d7e0 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:12.426319 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:12.448319 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:12.458209 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 128/245] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:12.587602 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700feed-4b02-48c2-ad43-d0d14e02d7e0 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:12.640468 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d4382-ca28-4752-ab41-7721a8c052a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:12.641111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d4382-ca28-4752-ab41-7721a8c052a6 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:12.641391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d4382-ca28-4752-ab41-7721a8c052a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:12.641664 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d4382-ca28-4752-ab41-7721a8c052a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:12.721165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32d4382-ca28-4752-ab41-7721a8c052a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:12.722640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32d4382-ca28-4752-ab41-7721a8c052a6 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:12.745231 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:12.772731 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9995099-3630-425d-9a0c-6a3da0ead000 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:12.847538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 118/246] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 00:07:12.854050 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d4382-ca28-4752-ab41-7721a8c052a6 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:12.917511 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9995099-3630-425d-9a0c-6a3da0ead000 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:12.968474 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edff55e-d9d3-40c1-a6fa-8de76737c281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:12.969141 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edff55e-d9d3-40c1-a6fa-8de76737c281 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:12.969423 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edff55e-d9d3-40c1-a6fa-8de76737c281 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:12.969696 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edff55e-d9d3-40c1-a6fa-8de76737c281 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:12.981734 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d4382-ca28-4752-ab41-7721a8c052a6 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:12.991413 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edff55e-d9d3-40c1-a6fa-8de76737c281 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:12.996654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-3edff55e-d9d3-40c1-a6fa-8de76737c281 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:12.996654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:12.996654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:12.996654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:12.996654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:12.996654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:12.996654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:12.996654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:12.996654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:12.996654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:12.996654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:12.996654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:12.996654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:12.996654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:12.996654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:12.996654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:12.997566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:12.997566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:12.997566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:12.997566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:12.997566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:12.997566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:12.997566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:12.997566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:12.997566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:12.997566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:12.997566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:12.997566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:12.997566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:12.997566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:12.997566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:12.997566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:12.998347 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:12.998347 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:12.998347 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:12.998347 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:12.998347 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:12.998347 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:12.998347 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:12.998347 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:12.998347 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:12.998347 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:12.998347 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:12.998347 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:12.998347 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 129/247] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:07:13.005117 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edff55e-d9d3-40c1-a6fa-8de76737c281 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:13.010320 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:13.033891 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f2baf-d024-491a-b526-592ef8d7c4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:13.034701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f2baf-d024-491a-b526-592ef8d7c4a1 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:13.035157 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f2baf-d024-491a-b526-592ef8d7c4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:13.035542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f2baf-d024-491a-b526-592ef8d7c4a1 None admin] PATH_INFO: `/v3/projects/a8b13074ade34e4d944896277be959f5` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:13.036798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f2baf-d024-491a-b526-592ef8d7c4a1 None admin] RBAC: Authorizing `identity:get_project(project_id=a8b13074ade34e4d944896277be959f5)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:13.038344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f2baf-d024-491a-b526-592ef8d7c4a1 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:13.045457 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 119/248] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:12 2022] GET /identity/v3/projects/a8b13074ade34e4d944896277be959f5 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:13.049202 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f2baf-d024-491a-b526-592ef8d7c4a1 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:13.080034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:13.161032 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edff55e-d9d3-40c1-a6fa-8de76737c281 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:13.181693 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f2baf-d024-491a-b526-592ef8d7c4a1 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:13.206214 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6e54e-4d64-43df-a694-a9a5c0b67f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:13.207977 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6e54e-4d64-43df-a694-a9a5c0b67f32 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:13.208402 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6e54e-4d64-43df-a694-a9a5c0b67f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:13.208941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6e54e-4d64-43df-a694-a9a5c0b67f32 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:13.209884 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6e54e-4d64-43df-a694-a9a5c0b67f32 None admin] RBAC: Adding query filter params (name=member) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:13.210567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6e54e-4d64-43df-a694-a9a5c0b67f32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:13.212763 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6e54e-4d64-43df-a694-a9a5c0b67f32 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:13.222836 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 130/249] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:07:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:13.223863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97262f-bd42-4818-a05c-59029e8f70f5 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:13.224146 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97262f-bd42-4818-a05c-59029e8f70f5 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:13.224475 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97262f-bd42-4818-a05c-59029e8f70f5 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:13.225860 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 131/250] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:13.227099 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e327d0b-a342-468a-b0ce-914e0442d0e6 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:13.230907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b1b99-e509-4e1e-bb20-d60dfb48e994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:13.231266 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e327d0b-a342-468a-b0ce-914e0442d0e6 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:13.231570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e327d0b-a342-468a-b0ce-914e0442d0e6 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:13.231689 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b1b99-e509-4e1e-bb20-d60dfb48e994 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:13.232552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b1b99-e509-4e1e-bb20-d60dfb48e994 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:13.232832 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b1b99-e509-4e1e-bb20-d60dfb48e994 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:13.233806 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 132/251] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:13.234135 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b1b99-e509-4e1e-bb20-d60dfb48e994 None admin] RBAC: Adding query filter params (role.id=65d36a76676c424db12c090f01595bdc, scope.project.id=a8b13074ade34e4d944896277be959f5, user.id=c9fcdd29f7d741a78e3fc08852ad17a3) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:13.234459 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b1b99-e509-4e1e-bb20-d60dfb48e994 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:13.234706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e327d0b-a342-468a-b0ce-914e0442d0e6 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:13.235967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b1b99-e509-4e1e-bb20-d60dfb48e994 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:13.246118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 120/252] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:07:13 2022] GET /identity/v3/role_assignments?user.id=c9fcdd29f7d741a78e3fc08852ad17a3&scope.project.id=a8b13074ade34e4d944896277be959f5&role.id=65d36a76676c424db12c090f01595bdc => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:13.249188 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba444009-c4d3-4457-a1c9-e06d10af84cc None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:13.249307 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba444009-c4d3-4457-a1c9-e06d10af84cc None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:13.249556 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba444009-c4d3-4457-a1c9-e06d10af84cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:13.287537 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-ba444009-c4d3-4457-a1c9-e06d10af84cc None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:13.289105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-ba444009-c4d3-4457-a1c9-e06d10af84cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:13.393412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:13.403141 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e327d0b-a342-468a-b0ce-914e0442d0e6 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:13.422206 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 121/253] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:13.423400 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e8143-5966-4b05-9d81-0d11bdb7f91f None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:13.423859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e8143-5966-4b05-9d81-0d11bdb7f91f None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:13.424309 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e8143-5966-4b05-9d81-0d11bdb7f91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:13.429129 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:13.444300 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-ff0e8143-5966-4b05-9d81-0d11bdb7f91f None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:13.445505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-ff0e8143-5966-4b05-9d81-0d11bdb7f91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:13.456863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0080f56-1f67-4016-a93c-faa8a75223aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:13.457505 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0080f56-1f67-4016-a93c-faa8a75223aa None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:13.457792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0080f56-1f67-4016-a93c-faa8a75223aa None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:13.458473 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0080f56-1f67-4016-a93c-faa8a75223aa None admin] PATH_INFO: `/v3/users/53d04e52fcba4793872c2660f8550a4d` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:13.460024 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0080f56-1f67-4016-a93c-faa8a75223aa None admin] RBAC: Authorizing `identity:get_user(user_id=53d04e52fcba4793872c2660f8550a4d)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:13.462300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0080f56-1f67-4016-a93c-faa8a75223aa None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:13.464722 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 133/254] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:07:13 2022] GET /identity/v3/users/53d04e52fcba4793872c2660f8550a4d => generated 297 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:13.470835 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0080f56-1f67-4016-a93c-faa8a75223aa None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:13.561776 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 122/255] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:13.619457 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0080f56-1f67-4016-a93c-faa8a75223aa None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:13.660817 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f57c7-4443-44e2-b6d4-b04f75c55813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:13.661806 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f57c7-4443-44e2-b6d4-b04f75c55813 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:13.662281 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f57c7-4443-44e2-b6d4-b04f75c55813 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:13.662776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f57c7-4443-44e2-b6d4-b04f75c55813 None admin] PATH_INFO: `/v3/projects/a8b13074ade34e4d944896277be959f5` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:13.664171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f57c7-4443-44e2-b6d4-b04f75c55813 None admin] RBAC: Authorizing `identity:get_project(project_id=a8b13074ade34e4d944896277be959f5)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:13.665933 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f57c7-4443-44e2-b6d4-b04f75c55813 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:13.668144 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 134/256] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:13 2022] GET /identity/v3/projects/a8b13074ade34e4d944896277be959f5 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:13.674002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e8143-5966-4b05-9d81-0d11bdb7f91f None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:13.787866 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e8143-5966-4b05-9d81-0d11bdb7f91f None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:13.816546 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:13.853125 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc792c-d0c6-4c89-ae72-e320d296e22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:13.853806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc792c-d0c6-4c89-ae72-e320d296e22b None admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:13.854088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc792c-d0c6-4c89-ae72-e320d296e22b None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:13.854362 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc792c-d0c6-4c89-ae72-e320d296e22b None admin] PATH_INFO: `/v3/projects/a8b13074ade34e4d944896277be959f5/users/53d04e52fcba4793872c2660f8550a4d/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:13.856344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afc792c-d0c6-4c89-ae72-e320d296e22b None admin] RBAC: Authorizing `identity:create_grant(project_id=a8b13074ade34e4d944896277be959f5, user_id=53d04e52fcba4793872c2660f8550a4d, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:13.858733 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afc792c-d0c6-4c89-ae72-e320d296e22b None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:13.860212 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f57c7-4443-44e2-b6d4-b04f75c55813 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:13.883434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 123/257] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 31 00:07:13 2022] PUT /identity/v3/projects/a8b13074ade34e4d944896277be959f5/users/53d04e52fcba4793872c2660f8550a4d/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:07:13.939870 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:14.025721 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f57c7-4443-44e2-b6d4-b04f75c55813 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:14.072801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a047c60-ea9b-4f37-82ba-294607f07f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:14.073801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a047c60-ea9b-4f37-82ba-294607f07f98 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:14.074283 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a047c60-ea9b-4f37-82ba-294607f07f98 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:14.074780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a047c60-ea9b-4f37-82ba-294607f07f98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:14.088172 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a047c60-ea9b-4f37-82ba-294607f07f98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:14.094828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-5a047c60-ea9b-4f37-82ba-294607f07f98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:07:14.094828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:14.094828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:14.094828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:14.094828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:14.094828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:14.094828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:14.094828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:14.094828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:14.094828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:14.094828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:14.094828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:14.094828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:14.094828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:14.094828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:14.094828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:14.095791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:14.095791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:14.095791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:14.095791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:14.095791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:14.095791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:14.095791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:14.095791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:14.095791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:14.095791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:14.095791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:14.095791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:14.095791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:14.095791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:14.095791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:14.095791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:14.096560 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:14.096560 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:14.096560 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:14.096560 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:14.096560 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:14.096560 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:14.096560 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:14.096560 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:14.096560 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:14.096560 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:14.096560 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:07:14.096560 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:14.098918 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 135/258] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:07:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:14.104410 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a047c60-ea9b-4f37-82ba-294607f07f98 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:14.104607 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc792c-d0c6-4c89-ae72-e320d296e22b None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:14.234076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a047c60-ea9b-4f37-82ba-294607f07f98 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:14.246245 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:14.271791 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc792c-d0c6-4c89-ae72-e320d296e22b None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:14.274518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19884004-2c4a-4664-93e1-13d845436756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:14.275416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19884004-2c4a-4664-93e1-13d845436756 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:14.275911 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19884004-2c4a-4664-93e1-13d845436756 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:14.276382 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19884004-2c4a-4664-93e1-13d845436756 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:14.277238 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19884004-2c4a-4664-93e1-13d845436756 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:14.277746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19884004-2c4a-4664-93e1-13d845436756 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:14.280036 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19884004-2c4a-4664-93e1-13d845436756 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:14.291742 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 136/259] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:14.299088 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19884004-2c4a-4664-93e1-13d845436756 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:14.340242 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e09ac8-4224-4a62-9bac-8b6bf88ac627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:14.340903 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e09ac8-4224-4a62-9bac-8b6bf88ac627 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:14.341220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e09ac8-4224-4a62-9bac-8b6bf88ac627 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:14.342156 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e09ac8-4224-4a62-9bac-8b6bf88ac627 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:14.368060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e09ac8-4224-4a62-9bac-8b6bf88ac627 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:14.373088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-e9e09ac8-4224-4a62-9bac-8b6bf88ac627 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:07:14.373088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:14.373088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:14.373088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:14.373088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:14.373088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:14.373088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:14.373088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:14.373088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:14.373088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:14.373088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:14.373088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:14.373088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:14.373088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:14.373088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:14.373088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:14.373964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:14.373964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:14.373964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:14.373964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:14.373964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:14.373964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:14.373964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:14.373964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:14.373964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:14.373964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:14.373964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:14.373964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:14.373964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:14.373964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:14.373964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:14.373964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:14.374751 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:14.374751 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:14.374751 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:14.374751 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:14.374751 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:14.374751 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:14.374751 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:14.374751 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:14.374751 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:14.374751 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:14.374751 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:07:14.374751 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:14.375665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 124/260] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 31 00:07:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:14.381855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e09ac8-4224-4a62-9bac-8b6bf88ac627 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:14.435438 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:14.450139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19884004-2c4a-4664-93e1-13d845436756 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:14.515862 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ed384-8923-447a-b8bc-9230945fdba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:14.516531 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ed384-8923-447a-b8bc-9230945fdba7 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:14.516820 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ed384-8923-447a-b8bc-9230945fdba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:14.516920 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e09ac8-4224-4a62-9bac-8b6bf88ac627 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:14.517092 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ed384-8923-447a-b8bc-9230945fdba7 None admin] PATH_INFO: `/v3/users/9c58d322d6f84ed5a67aafe90ebfb284` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:14.518390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ed384-8923-447a-b8bc-9230945fdba7 None admin] RBAC: Authorizing `identity:get_user(user_id=9c58d322d6f84ed5a67aafe90ebfb284)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:14.519925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ed384-8923-447a-b8bc-9230945fdba7 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:14.521973 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 137/261] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:07:14 2022] GET /identity/v3/users/9c58d322d6f84ed5a67aafe90ebfb284 => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:14.527588 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ed384-8923-447a-b8bc-9230945fdba7 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:14.557805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccddd3d-8b4a-4117-9b5f-91dadb63ca10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:14.558423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccddd3d-8b4a-4117-9b5f-91dadb63ca10 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:14.558702 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccddd3d-8b4a-4117-9b5f-91dadb63ca10 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:14.558969 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccddd3d-8b4a-4117-9b5f-91dadb63ca10 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:14.559784 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccddd3d-8b4a-4117-9b5f-91dadb63ca10 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:14.560759 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccddd3d-8b4a-4117-9b5f-91dadb63ca10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:14.562131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccddd3d-8b4a-4117-9b5f-91dadb63ca10 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:14.571099 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 125/262] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:07:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:14.579874 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccddd3d-8b4a-4117-9b5f-91dadb63ca10 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:14.581203 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:14.674555 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ed384-8923-447a-b8bc-9230945fdba7 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:14.681624 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:14.691962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccddd3d-8b4a-4117-9b5f-91dadb63ca10 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:14.731956 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb843761-9668-4091-85f5-a140053a178d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:14.732986 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0138ea1-d04d-4aba-ad0e-ff4967af8e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:14.733867 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0138ea1-d04d-4aba-ad0e-ff4967af8e6c None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:14.734755 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb843761-9668-4091-85f5-a140053a178d None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:14.735275 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0138ea1-d04d-4aba-ad0e-ff4967af8e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:14.735744 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb843761-9668-4091-85f5-a140053a178d None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:14.736221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0138ea1-d04d-4aba-ad0e-ff4967af8e6c None admin] PATH_INFO: `/v3/users/9c58d322d6f84ed5a67aafe90ebfb284` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:14.736680 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb843761-9668-4091-85f5-a140053a178d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:14.738520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0138ea1-d04d-4aba-ad0e-ff4967af8e6c None admin] RBAC: Authorizing `identity:get_user(user_id=9c58d322d6f84ed5a67aafe90ebfb284)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:14.740529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0138ea1-d04d-4aba-ad0e-ff4967af8e6c None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:14.744475 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 126/263] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:07:14 2022] GET /identity/v3/users/9c58d322d6f84ed5a67aafe90ebfb284 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:14.755900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0138ea1-d04d-4aba-ad0e-ff4967af8e6c None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:14.891108 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb843761-9668-4091-85f5-a140053a178d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:14.893699 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb843761-9668-4091-85f5-a140053a178d None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:14.944753 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:14.982953 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0138ea1-d04d-4aba-ad0e-ff4967af8e6c None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:15.022545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 138/264] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:15.028433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb843761-9668-4091-85f5-a140053a178d None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:15.063323 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70639ba9-3f65-425e-88b1-3c6a20c85eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:15.063969 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70639ba9-3f65-425e-88b1-3c6a20c85eec None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:15.064256 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70639ba9-3f65-425e-88b1-3c6a20c85eec None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:15.064512 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70639ba9-3f65-425e-88b1-3c6a20c85eec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:15.128351 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:15.194711 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70639ba9-3f65-425e-88b1-3c6a20c85eec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:15.196675 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70639ba9-3f65-425e-88b1-3c6a20c85eec None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:15.197886 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb843761-9668-4091-85f5-a140053a178d None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:15.264266 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2933e53-0628-4c5c-8cb8-58723a69ab8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:15.264888 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2933e53-0628-4c5c-8cb8-58723a69ab8c None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:15.265263 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2933e53-0628-4c5c-8cb8-58723a69ab8c None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:15.265452 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2933e53-0628-4c5c-8cb8-58723a69ab8c None admin] PATH_INFO: `/v3/projects/a8b13074ade34e4d944896277be959f5` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:15.266607 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2933e53-0628-4c5c-8cb8-58723a69ab8c None admin] RBAC: Authorizing `identity:get_project(project_id=a8b13074ade34e4d944896277be959f5)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:15.268111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2933e53-0628-4c5c-8cb8-58723a69ab8c None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:15.270216 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 139/265] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:15 2022] GET /identity/v3/projects/a8b13074ade34e4d944896277be959f5 => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:15.275910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2933e53-0628-4c5c-8cb8-58723a69ab8c None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:15.298280 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:15.313896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 127/266] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:15.319292 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70639ba9-3f65-425e-88b1-3c6a20c85eec None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:15.419943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:15.459430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70639ba9-3f65-425e-88b1-3c6a20c85eec None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:15.478819 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2933e53-0628-4c5c-8cb8-58723a69ab8c None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:15.499275 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e4343-3c9b-437c-a041-bd79c2ec46ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:15.500502 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e4343-3c9b-437c-a041-bd79c2ec46ca None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:15.501028 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e4343-3c9b-437c-a041-bd79c2ec46ca None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:15.501509 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e4343-3c9b-437c-a041-bd79c2ec46ca None admin] PATH_INFO: `/v3/projects/a8b13074ade34e4d944896277be959f5` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:15.502935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592e4343-3c9b-437c-a041-bd79c2ec46ca None admin] RBAC: Authorizing `identity:get_project(project_id=a8b13074ade34e4d944896277be959f5)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:15.504890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592e4343-3c9b-437c-a041-bd79c2ec46ca None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:15.507233 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 128/267] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:15 2022] GET /identity/v3/projects/a8b13074ade34e4d944896277be959f5 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:15.513994 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e4343-3c9b-437c-a041-bd79c2ec46ca None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:15.553776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c73275-3b8a-492b-92f1-72905639ab02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:15.554758 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c73275-3b8a-492b-92f1-72905639ab02 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:15.555290 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c73275-3b8a-492b-92f1-72905639ab02 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:15.555757 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c73275-3b8a-492b-92f1-72905639ab02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:15.556685 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c73275-3b8a-492b-92f1-72905639ab02 None admin] RBAC: Adding query filter params (role.id=34df06ce84ad4cf8b3bd1a4490228017, scope.project.id=a8b13074ade34e4d944896277be959f5, user.id=9c58d322d6f84ed5a67aafe90ebfb284) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:15.557193 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c73275-3b8a-492b-92f1-72905639ab02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:15.558849 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c73275-3b8a-492b-92f1-72905639ab02 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:15.573178 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 140/268] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:07:15 2022] GET /identity/v3/role_assignments?user.id=9c58d322d6f84ed5a67aafe90ebfb284&scope.project.id=a8b13074ade34e4d944896277be959f5&role.id=34df06ce84ad4cf8b3bd1a4490228017 => generated 608 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:15.627692 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e45c4-8c1f-4fa8-bbf0-752b1574a62d None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:15.628307 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e45c4-8c1f-4fa8-bbf0-752b1574a62d None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:15.628775 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e45c4-8c1f-4fa8-bbf0-752b1574a62d None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:15.630665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 141/269] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:07:15.636582 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0938f8-2cad-4600-a1df-c8ce54466ac5 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:15.636768 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0938f8-2cad-4600-a1df-c8ce54466ac5 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:15.637012 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0938f8-2cad-4600-a1df-c8ce54466ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:15.660365 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e4343-3c9b-437c-a041-bd79c2ec46ca None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:15.691336 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-eb0938f8-2cad-4600-a1df-c8ce54466ac5 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:15.692592 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-eb0938f8-2cad-4600-a1df-c8ce54466ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:15.695430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:15.705564 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010045a5-b641-4316-8ef6-b399394ec0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:15.706561 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010045a5-b641-4316-8ef6-b399394ec0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:15.707073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010045a5-b641-4316-8ef6-b399394ec0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:15.707576 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010045a5-b641-4316-8ef6-b399394ec0e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:15.708654 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010045a5-b641-4316-8ef6-b399394ec0e2 None admin] RBAC: Adding query filter params (role.id=4d91c00f40304b9e9ef4e6c89f46e5c5, scope.project.id=a8b13074ade34e4d944896277be959f5, user.id=9c58d322d6f84ed5a67aafe90ebfb284) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:15.709181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010045a5-b641-4316-8ef6-b399394ec0e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:15.711015 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010045a5-b641-4316-8ef6-b399394ec0e2 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:15.718392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 129/270] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:07:15 2022] GET /identity/v3/role_assignments?user.id=9c58d322d6f84ed5a67aafe90ebfb284&scope.project.id=a8b13074ade34e4d944896277be959f5&role.id=4d91c00f40304b9e9ef4e6c89f46e5c5 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:15.815868 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 142/271] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:16.210941 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010045a5-b641-4316-8ef6-b399394ec0e2 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:16.233323 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43afbf8-3667-44bb-a978-603634c3fe7e None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:16.233639 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43afbf8-3667-44bb-a978-603634c3fe7e None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:16.234033 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43afbf8-3667-44bb-a978-603634c3fe7e None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:16.236458 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 143/272] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:16.243558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d52fc-ce8b-4303-873c-ffc5c45a14de None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:16.243738 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d52fc-ce8b-4303-873c-ffc5c45a14de None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:16.244021 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d52fc-ce8b-4303-873c-ffc5c45a14de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:16.259264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:16.277117 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-ef9d52fc-ce8b-4303-873c-ffc5c45a14de None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:16.278317 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-ef9d52fc-ce8b-4303-873c-ffc5c45a14de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:16.358115 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010045a5-b641-4316-8ef6-b399394ec0e2 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:16.371620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 144/273] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:16.373228 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7472dab-452c-4c3b-a30f-d972425e2622 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:16.373516 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7472dab-452c-4c3b-a30f-d972425e2622 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:16.373793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7472dab-452c-4c3b-a30f-d972425e2622 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:16.379069 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 145/274] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:16.384552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ee114-9f06-4d6d-ba89-498cb5bf1bfb None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:16.384847 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ee114-9f06-4d6d-ba89-498cb5bf1bfb None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:16.385125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ee114-9f06-4d6d-ba89-498cb5bf1bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:16.401433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638c4f5-530d-4b34-a593-90e9378ffea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:16.402373 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638c4f5-530d-4b34-a593-90e9378ffea5 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:16.402849 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638c4f5-530d-4b34-a593-90e9378ffea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:16.403419 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638c4f5-530d-4b34-a593-90e9378ffea5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:16.409486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-6e0ee114-9f06-4d6d-ba89-498cb5bf1bfb None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:16.410922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-6e0ee114-9f06-4d6d-ba89-498cb5bf1bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:16.416221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638c4f5-530d-4b34-a593-90e9378ffea5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:16.421981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-3638c4f5-530d-4b34-a593-90e9378ffea5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:16.421981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:16.421981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:16.421981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:16.421981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:16.421981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:16.421981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:16.421981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:16.421981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:16.421981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:16.421981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:16.421981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:16.421981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:16.421981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:16.421981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:16.421981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:16.422870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:16.422870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:16.422870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:16.422870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:16.422870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:16.422870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:16.422870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:16.422870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:16.422870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:16.422870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:16.422870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:16.422870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:16.422870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:16.422870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:16.422870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:16.422870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:16.423968 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:16.423968 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:16.423968 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:16.423968 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:16.423968 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:16.423968 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:16.423968 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:16.423968 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:16.423968 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:16.423968 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:16.423968 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:16.423968 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:16.426351 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 130/275] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:16.432501 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638c4f5-530d-4b34-a593-90e9378ffea5 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:16.499154 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 146/276] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:16.511399 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:16.562798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638c4f5-530d-4b34-a593-90e9378ffea5 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:16.597414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ccfd7-0481-42db-a5fd-b319f7510453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:16.598027 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ccfd7-0481-42db-a5fd-b319f7510453 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:16.598308 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ccfd7-0481-42db-a5fd-b319f7510453 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:16.598580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ccfd7-0481-42db-a5fd-b319f7510453 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:16.599323 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ccfd7-0481-42db-a5fd-b319f7510453 None admin] RBAC: Adding query filter params (name=member) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:16.599759 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ccfd7-0481-42db-a5fd-b319f7510453 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:16.601205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ccfd7-0481-42db-a5fd-b319f7510453 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:16.610164 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 131/277] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:07:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:16.616115 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ee114-9f06-4d6d-ba89-498cb5bf1bfb None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:16.662648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ccfd7-0481-42db-a5fd-b319f7510453 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:16.682746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:16.729146 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ee114-9f06-4d6d-ba89-498cb5bf1bfb None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:16.768437 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c226cf-9319-4a71-9424-fada146adcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:16.768976 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c226cf-9319-4a71-9424-fada146adcb9 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:16.769258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c226cf-9319-4a71-9424-fada146adcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:16.769521 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c226cf-9319-4a71-9424-fada146adcb9 None admin] PATH_INFO: `/v3/users/996eaf17b7f04faa905120a047d7f5b2` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:16.771714 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ccfd7-0481-42db-a5fd-b319f7510453 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:16.783030 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c226cf-9319-4a71-9424-fada146adcb9 None admin] RBAC: Authorizing `identity:get_user(user_id=996eaf17b7f04faa905120a047d7f5b2)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:16.784570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c226cf-9319-4a71-9424-fada146adcb9 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:16.786400 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 147/278] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:07:16 2022] GET /identity/v3/users/996eaf17b7f04faa905120a047d7f5b2 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:16.787582 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f632e7-0e63-4f04-93e6-88e959a22aa1 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:16.787913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f632e7-0e63-4f04-93e6-88e959a22aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:16.788157 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f632e7-0e63-4f04-93e6-88e959a22aa1 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:16.788473 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:16.789698 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 148/279] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:16.790196 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f632e7-0e63-4f04-93e6-88e959a22aa1 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:16.811504 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ebec4-bb79-4d64-9fc3-8ce4c77c0968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:16.812135 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ebec4-bb79-4d64-9fc3-8ce4c77c0968 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:16.812445 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ebec4-bb79-4d64-9fc3-8ce4c77c0968 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:16.812717 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ebec4-bb79-4d64-9fc3-8ce4c77c0968 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:16.822759 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ebec4-bb79-4d64-9fc3-8ce4c77c0968 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:16.828547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-ed4ebec4-bb79-4d64-9fc3-8ce4c77c0968 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:07:16.828547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:16.828547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:16.828547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:16.828547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:16.828547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:16.828547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:16.828547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:16.828547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:16.828547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:16.828547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:16.828547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:16.828547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:16.828547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:16.828547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:16.828547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:16.829462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:16.829462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:16.829462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:16.829462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:16.829462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:16.829462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:16.829462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:16.829462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:16.829462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:16.829462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:16.829462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:16.829462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:16.829462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:16.829462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:16.829462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:16.829462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:16.830246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:16.830246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:16.830246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:16.830246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:16.830246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:16.830246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:16.830246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:16.830246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:16.830246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:16.830246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:16.830246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:07:16.830246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:16.830246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 132/280] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:07:16.830896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ebec4-bb79-4d64-9fc3-8ce4c77c0968 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:16.894993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f632e7-0e63-4f04-93e6-88e959a22aa1 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:16.914678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:16.928892 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f05f4-3fb8-4d4e-84a7-77d1de2cc658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:16.929523 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f05f4-3fb8-4d4e-84a7-77d1de2cc658 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:16.929800 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f05f4-3fb8-4d4e-84a7-77d1de2cc658 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:16.930072 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f05f4-3fb8-4d4e-84a7-77d1de2cc658 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:16.930841 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ebec4-bb79-4d64-9fc3-8ce4c77c0968 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:16.931187 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f05f4-3fb8-4d4e-84a7-77d1de2cc658 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:16.932622 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f05f4-3fb8-4d4e-84a7-77d1de2cc658 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:16.934359 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 149/281] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:16.935900 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f21b53-01bf-477e-821b-9c497ba75ed6 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:16.936186 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f21b53-01bf-477e-821b-9c497ba75ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:16.936555 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f21b53-01bf-477e-821b-9c497ba75ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:16.954464 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-65f21b53-01bf-477e-821b-9c497ba75ed6 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:16.955677 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-65f21b53-01bf-477e-821b-9c497ba75ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:16.967012 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21619a-0c7a-47a6-9c2a-93d9a81b266d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:16.967656 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21619a-0c7a-47a6-9c2a-93d9a81b266d None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:16.967937 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21619a-0c7a-47a6-9c2a-93d9a81b266d None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:16.968209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21619a-0c7a-47a6-9c2a-93d9a81b266d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:16.968942 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce21619a-0c7a-47a6-9c2a-93d9a81b266d None admin] RBAC: Adding query filter params (role.id=cc8bd97c7b9745d1a52746650a2e49b9, user.id=996eaf17b7f04faa905120a047d7f5b2, scope.system=all) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:16.969253 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce21619a-0c7a-47a6-9c2a-93d9a81b266d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:16.970709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce21619a-0c7a-47a6-9c2a-93d9a81b266d None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:16.975909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 133/282] 10.222.0.67 () {56 vars in 1308 bytes} [Wed Aug 31 00:07:16 2022] GET /identity/v3/role_assignments?user.id=996eaf17b7f04faa905120a047d7f5b2&scope.system=all&role.id=cc8bd97c7b9745d1a52746650a2e49b9 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:16.976692 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21619a-0c7a-47a6-9c2a-93d9a81b266d None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:17.024810 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:17.055069 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 150/283] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:17.055069 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f21b53-01bf-477e-821b-9c497ba75ed6 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:17.104410 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21619a-0c7a-47a6-9c2a-93d9a81b266d None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:17.146410 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe4a48-5f2b-45f3-b493-ef5a77362d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:17.147210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe4a48-5f2b-45f3-b493-ef5a77362d95 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:17.147583 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe4a48-5f2b-45f3-b493-ef5a77362d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:17.147943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe4a48-5f2b-45f3-b493-ef5a77362d95 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:17.148709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe4a48-5f2b-45f3-b493-ef5a77362d95 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:17.149198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe4a48-5f2b-45f3-b493-ef5a77362d95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:17.150752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe4a48-5f2b-45f3-b493-ef5a77362d95 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:17.159455 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 134/284] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:07:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:17.165381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe4a48-5f2b-45f3-b493-ef5a77362d95 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:17.181975 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f21b53-01bf-477e-821b-9c497ba75ed6 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:17.205920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:17.229508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71930f-fe51-41eb-9a25-ab8a8ddcda2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:17.230285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71930f-fe51-41eb-9a25-ab8a8ddcda2d None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:17.230874 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71930f-fe51-41eb-9a25-ab8a8ddcda2d None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:17.231198 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71930f-fe51-41eb-9a25-ab8a8ddcda2d None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:17.232496 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71930f-fe51-41eb-9a25-ab8a8ddcda2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:17.234979 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71930f-fe51-41eb-9a25-ab8a8ddcda2d None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:17.266679 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 151/285] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 31 00:07:17 2022] POST /identity/v3/users => generated 293 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:07:17.295458 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:17.303095 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe4a48-5f2b-45f3-b493-ef5a77362d95 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:17.346753 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25755e-80cc-4fad-bbd8-aa4a7aaaaa53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:17.347797 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25755e-80cc-4fad-bbd8-aa4a7aaaaa53 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:17.348286 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25755e-80cc-4fad-bbd8-aa4a7aaaaa53 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:17.348786 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25755e-80cc-4fad-bbd8-aa4a7aaaaa53 None admin] PATH_INFO: `/v3/users/c9fcdd29f7d741a78e3fc08852ad17a3` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:17.350306 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25755e-80cc-4fad-bbd8-aa4a7aaaaa53 None admin] RBAC: Authorizing `identity:get_user(user_id=c9fcdd29f7d741a78e3fc08852ad17a3)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:17.352071 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25755e-80cc-4fad-bbd8-aa4a7aaaaa53 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:17.352827 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71930f-fe51-41eb-9a25-ab8a8ddcda2d None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:17.355175 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 135/286] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:07:17 2022] GET /identity/v3/users/c9fcdd29f7d741a78e3fc08852ad17a3 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:17.366249 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25755e-80cc-4fad-bbd8-aa4a7aaaaa53 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:17.476666 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71930f-fe51-41eb-9a25-ab8a8ddcda2d None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:17.498072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25755e-80cc-4fad-bbd8-aa4a7aaaaa53 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:17.521468 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493beadc-8768-4967-8a80-0ef6c4d00c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:17.522116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493beadc-8768-4967-8a80-0ef6c4d00c38 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:17.522406 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493beadc-8768-4967-8a80-0ef6c4d00c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:17.522709 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493beadc-8768-4967-8a80-0ef6c4d00c38 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:17.537249 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493beadc-8768-4967-8a80-0ef6c4d00c38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:17.542526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-493beadc-8768-4967-8a80-0ef6c4d00c38 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:17.542526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:17.542526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:17.542526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:17.542526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:17.542526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:17.542526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:17.542526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:17.542526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:17.542526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:17.542526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:17.542526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:17.542526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:17.542526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:17.542526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:17.542526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:17.543585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:17.543585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:17.543585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:17.543585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:17.543585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:17.543585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:17.543585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:17.543585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:17.543585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:17.543585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:17.543585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:17.543585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:17.543585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:17.543585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:17.543585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:17.543585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:17.544912 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:17.544912 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:17.544912 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:17.544912 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:17.544912 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:17.544912 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:17.544912 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:17.544912 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:17.544912 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:17.544912 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:17.544912 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:17.544912 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:17.545658 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 152/287] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:07:17.551158 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493beadc-8768-4967-8a80-0ef6c4d00c38 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:17.555377 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac6ca2-c955-4870-812c-df3f51a6acd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:17.555969 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac6ca2-c955-4870-812c-df3f51a6acd9 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:17.556246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac6ca2-c955-4870-812c-df3f51a6acd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:17.556516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac6ca2-c955-4870-812c-df3f51a6acd9 None admin] PATH_INFO: `/v3/projects/a8b13074ade34e4d944896277be959f5` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:17.559771 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac6ca2-c955-4870-812c-df3f51a6acd9 None admin] RBAC: Authorizing `identity:get_project(project_id=a8b13074ade34e4d944896277be959f5)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:17.562271 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac6ca2-c955-4870-812c-df3f51a6acd9 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:17.569010 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 136/288] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:17 2022] GET /identity/v3/projects/a8b13074ade34e4d944896277be959f5 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:17.570464 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac6ca2-c955-4870-812c-df3f51a6acd9 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:17.592363 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:17.613341 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:17.678153 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493beadc-8768-4967-8a80-0ef6c4d00c38 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:17.702195 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac6ca2-c955-4870-812c-df3f51a6acd9 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:17.714769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9347e0-e212-4fab-8828-e21a4b7b7b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:17.715410 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9347e0-e212-4fab-8828-e21a4b7b7b6d None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:17.715685 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9347e0-e212-4fab-8828-e21a4b7b7b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:17.716015 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9347e0-e212-4fab-8828-e21a4b7b7b6d None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:17.718720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9347e0-e212-4fab-8828-e21a4b7b7b6d None admin] RBAC: Adding query filter params (name=member) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:17.718720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9347e0-e212-4fab-8828-e21a4b7b7b6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:17.719072 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9347e0-e212-4fab-8828-e21a4b7b7b6d None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:17.729032 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 153/289] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:07:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:17.735499 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9347e0-e212-4fab-8828-e21a4b7b7b6d None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:17.746694 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a4fae-2a54-47ba-bdf6-cfcfe35a70ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:17.747915 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a4fae-2a54-47ba-bdf6-cfcfe35a70ae None admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:17.748737 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a4fae-2a54-47ba-bdf6-cfcfe35a70ae None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:17.749088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a4fae-2a54-47ba-bdf6-cfcfe35a70ae None admin] PATH_INFO: `/v3/projects/a8b13074ade34e4d944896277be959f5/users/c9fcdd29f7d741a78e3fc08852ad17a3/roles/65d36a76676c424db12c090f01595bdc` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:17.752184 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15a4fae-2a54-47ba-bdf6-cfcfe35a70ae None admin] RBAC: Authorizing `identity:create_grant(project_id=a8b13074ade34e4d944896277be959f5, user_id=c9fcdd29f7d741a78e3fc08852ad17a3, role_id=65d36a76676c424db12c090f01595bdc)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:17.753641 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15a4fae-2a54-47ba-bdf6-cfcfe35a70ae None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:17.779884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 137/290] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 31 00:07:17 2022] PUT /identity/v3/projects/a8b13074ade34e4d944896277be959f5/users/c9fcdd29f7d741a78e3fc08852ad17a3/roles/65d36a76676c424db12c090f01595bdc => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:07:17.876383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:17.887506 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9347e0-e212-4fab-8828-e21a4b7b7b6d None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:17.925529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d3449d-1fbc-47d0-9de6-1223432fc534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:17.926518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3449d-1fbc-47d0-9de6-1223432fc534 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:17.926990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3449d-1fbc-47d0-9de6-1223432fc534 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:17.927494 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3449d-1fbc-47d0-9de6-1223432fc534 None admin] PATH_INFO: `/v3/users/53d04e52fcba4793872c2660f8550a4d` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:17.928921 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d3449d-1fbc-47d0-9de6-1223432fc534 None admin] RBAC: Authorizing `identity:get_user(user_id=53d04e52fcba4793872c2660f8550a4d)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:17.930676 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d3449d-1fbc-47d0-9de6-1223432fc534 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:17.932915 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 154/291] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:07:17 2022] GET /identity/v3/users/53d04e52fcba4793872c2660f8550a4d => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:17.939707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a4fae-2a54-47ba-bdf6-cfcfe35a70ae None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:18.065016 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:18.074018 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a4fae-2a54-47ba-bdf6-cfcfe35a70ae None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:18.116084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49779be-e945-48c2-86f9-97615cca08db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:18.116715 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49779be-e945-48c2-86f9-97615cca08db None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:18.116997 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49779be-e945-48c2-86f9-97615cca08db None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:18.117271 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49779be-e945-48c2-86f9-97615cca08db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:18.219657 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49779be-e945-48c2-86f9-97615cca08db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:18.221057 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49779be-e945-48c2-86f9-97615cca08db None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:18.341288 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:18.347883 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 138/292] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:18.353281 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d3449d-1fbc-47d0-9de6-1223432fc534 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:18.524500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d3449d-1fbc-47d0-9de6-1223432fc534 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:18.570958 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa0214a-9b61-46a8-be8f-9818093a47d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:18.571630 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0214a-9b61-46a8-be8f-9818093a47d0 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:18.571911 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0214a-9b61-46a8-be8f-9818093a47d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:18.572183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0214a-9b61-46a8-be8f-9818093a47d0 None admin] PATH_INFO: `/v3/projects/a8b13074ade34e4d944896277be959f5` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:18.573323 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa0214a-9b61-46a8-be8f-9818093a47d0 None admin] RBAC: Authorizing `identity:get_project(project_id=a8b13074ade34e4d944896277be959f5)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:18.574777 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa0214a-9b61-46a8-be8f-9818093a47d0 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:18.577681 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 155/293] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:18 2022] GET /identity/v3/projects/a8b13074ade34e4d944896277be959f5 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:18.584371 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49779be-e945-48c2-86f9-97615cca08db None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:18.719832 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49779be-e945-48c2-86f9-97615cca08db None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:18.759631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d428356-2bd1-4537-9d03-aeefc05b16ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:18.760300 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d428356-2bd1-4537-9d03-aeefc05b16ef None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:18.760581 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d428356-2bd1-4537-9d03-aeefc05b16ef None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:18.761129 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d428356-2bd1-4537-9d03-aeefc05b16ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:18.761860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d428356-2bd1-4537-9d03-aeefc05b16ef None admin] RBAC: Adding query filter params (role.id=cc8bd97c7b9745d1a52746650a2e49b9, scope.project.id=a8b13074ade34e4d944896277be959f5, user.id=53d04e52fcba4793872c2660f8550a4d) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:18.762163 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d428356-2bd1-4537-9d03-aeefc05b16ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:18.763728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d428356-2bd1-4537-9d03-aeefc05b16ef None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:18.769911 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 139/294] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:07:18 2022] GET /identity/v3/role_assignments?user.id=53d04e52fcba4793872c2660f8550a4d&scope.project.id=a8b13074ade34e4d944896277be959f5&role.id=cc8bd97c7b9745d1a52746650a2e49b9 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:19.280995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae3cb6-9914-48d2-b71d-24205c144bd1 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:19.281643 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae3cb6-9914-48d2-b71d-24205c144bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:19.282183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae3cb6-9914-48d2-b71d-24205c144bd1 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:19.283984 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 156/295] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:19.290184 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e0215-a228-4354-b5cc-f29cb95fcd65 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:19.290489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e0215-a228-4354-b5cc-f29cb95fcd65 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:19.290819 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e0215-a228-4354-b5cc-f29cb95fcd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:19.315213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-9f0e0215-a228-4354-b5cc-f29cb95fcd65 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:19.316727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-9f0e0215-a228-4354-b5cc-f29cb95fcd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:19.383242 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:19.403399 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 140/296] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:19.687100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae3cb6-9914-48d2-b71d-24205c144bd1 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:19.721093 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:19.761158 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f4760-550e-4288-9a6f-f204cddd2ff9 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:19.761666 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f4760-550e-4288-9a6f-f204cddd2ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:19.761741 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f4760-550e-4288-9a6f-f204cddd2ff9 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:19.763133 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 141/297] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:07:19.768415 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891f0d0-fb08-4750-a290-0fab69e60d5b None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:19.768954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891f0d0-fb08-4750-a290-0fab69e60d5b None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:19.769419 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891f0d0-fb08-4750-a290-0fab69e60d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:19.791136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-e891f0d0-fb08-4750-a290-0fab69e60d5b None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:19.792729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-e891f0d0-fb08-4750-a290-0fab69e60d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:19.802257 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae3cb6-9914-48d2-b71d-24205c144bd1 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:19.845851 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b6a4c-13c5-4d6a-ad10-1e983fb75c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:19.846488 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b6a4c-13c5-4d6a-ad10-1e983fb75c1d None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:19.846766 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b6a4c-13c5-4d6a-ad10-1e983fb75c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:19.847068 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b6a4c-13c5-4d6a-ad10-1e983fb75c1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:19.864937 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b6a4c-13c5-4d6a-ad10-1e983fb75c1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:19.871941 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 142/298] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:07:19.872098 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-7f4b6a4c-13c5-4d6a-ad10-1e983fb75c1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:19.872098 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:19.872098 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:19.872098 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:19.872098 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:19.872098 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:19.872098 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:19.872098 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:19.872098 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:19.872098 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:19.872098 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:19.872098 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:19.872098 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:19.872098 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:19.872098 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:19.872098 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:19.873831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:19.873831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:19.873831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:19.873831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:19.873831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:19.873831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:19.873831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:19.873831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:19.873831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:19.873831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:19.873831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:19.873831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:19.873831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:19.873831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:19.873831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:19.873831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:19.881946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:19.881946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:19.881946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:19.881946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:19.881946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:19.881946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:19.881946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:19.881946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:19.881946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:19.881946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:19.881946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:19.881946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:19.881946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 157/299] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:19.881946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b6a4c-13c5-4d6a-ad10-1e983fb75c1d None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:19.979604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:19.985772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b6a4c-13c5-4d6a-ad10-1e983fb75c1d None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:20.003939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16979e-b9c3-4b00-97b5-a775c1fc613a None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:20.004231 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16979e-b9c3-4b00-97b5-a775c1fc613a None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:20.004510 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16979e-b9c3-4b00-97b5-a775c1fc613a None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:20.005856 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 143/300] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:20.010802 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372c5ed-b8dd-479e-8b7c-dbb41d411b1d None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:20.011115 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372c5ed-b8dd-479e-8b7c-dbb41d411b1d None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:20.011396 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372c5ed-b8dd-479e-8b7c-dbb41d411b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:20.023208 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034ac2c-0c30-4d24-a136-0cbb1159d15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:20.023823 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034ac2c-0c30-4d24-a136-0cbb1159d15d None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:20.024602 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034ac2c-0c30-4d24-a136-0cbb1159d15d None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:20.024887 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034ac2c-0c30-4d24-a136-0cbb1159d15d None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:20.025560 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034ac2c-0c30-4d24-a136-0cbb1159d15d None admin] RBAC: Adding query filter params (name=member) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:20.025866 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034ac2c-0c30-4d24-a136-0cbb1159d15d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:20.027687 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034ac2c-0c30-4d24-a136-0cbb1159d15d None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:20.031111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-6372c5ed-b8dd-479e-8b7c-dbb41d411b1d None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:20.032310 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-6372c5ed-b8dd-479e-8b7c-dbb41d411b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:20.037942 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 158/301] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:07:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:20.043895 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034ac2c-0c30-4d24-a136-0cbb1159d15d None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:20.115307 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 144/302] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:20.126520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d34aeb-8611-46c6-8a07-7a094d6b5567 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:20.126818 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d34aeb-8611-46c6-8a07-7a094d6b5567 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:20.127093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d34aeb-8611-46c6-8a07-7a094d6b5567 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:20.128536 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 145/303] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:07:20.133539 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9adba-e040-4158-aa3b-c9cd018b49e4 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:20.133824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9adba-e040-4158-aa3b-c9cd018b49e4 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:20.134097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9adba-e040-4158-aa3b-c9cd018b49e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:20.152487 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-b2e9adba-e040-4158-aa3b-c9cd018b49e4 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:20.154232 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b2e9adba-e040-4158-aa3b-c9cd018b49e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:20.190118 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034ac2c-0c30-4d24-a136-0cbb1159d15d None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:20.207318 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:20.228157 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843c376-65dc-4185-afb0-65c598ff9bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:20.228919 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843c376-65dc-4185-afb0-65c598ff9bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:20.229075 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843c376-65dc-4185-afb0-65c598ff9bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:20.229340 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843c376-65dc-4185-afb0-65c598ff9bd4 None admin] PATH_INFO: `/v3/users/996eaf17b7f04faa905120a047d7f5b2` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:20.230520 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843c376-65dc-4185-afb0-65c598ff9bd4 None admin] RBAC: Authorizing `identity:get_user(user_id=996eaf17b7f04faa905120a047d7f5b2)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:20.231999 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843c376-65dc-4185-afb0-65c598ff9bd4 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:20.233765 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 159/304] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:07:20 2022] GET /identity/v3/users/996eaf17b7f04faa905120a047d7f5b2 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:20.234336 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843c376-65dc-4185-afb0-65c598ff9bd4 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:20.244954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 146/305] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:20.245166 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e9adba-e040-4158-aa3b-c9cd018b49e4 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:20.271078 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:20.355814 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843c376-65dc-4185-afb0-65c598ff9bd4 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:20.369558 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e9adba-e040-4158-aa3b-c9cd018b49e4 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:20.396588 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cf3b9-5d5f-4f72-9bca-024d4822622e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:20.397345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cf3b9-5d5f-4f72-9bca-024d4822622e None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:20.397625 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cf3b9-5d5f-4f72-9bca-024d4822622e None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:20.397898 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cf3b9-5d5f-4f72-9bca-024d4822622e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:20.398998 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78cf3b9-5d5f-4f72-9bca-024d4822622e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:20.400739 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78cf3b9-5d5f-4f72-9bca-024d4822622e None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:20.402304 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 160/306] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:20.408317 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cf3b9-5d5f-4f72-9bca-024d4822622e None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:20.409610 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd269e-81ae-487a-88fa-404760235601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:20.410317 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd269e-81ae-487a-88fa-404760235601 None admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:20.410604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd269e-81ae-487a-88fa-404760235601 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:20.410931 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd269e-81ae-487a-88fa-404760235601 None admin] PATH_INFO: `/v3/system/users/996eaf17b7f04faa905120a047d7f5b2/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:20.412677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd269e-81ae-487a-88fa-404760235601 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=996eaf17b7f04faa905120a047d7f5b2, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:20.415288 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd269e-81ae-487a-88fa-404760235601 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:20.425656 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 147/307] 10.222.0.67 () {58 vars in 1334 bytes} [Wed Aug 31 00:07:20 2022] PUT /identity/v3/system/users/996eaf17b7f04faa905120a047d7f5b2/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:07:20.426748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd269e-81ae-487a-88fa-404760235601 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:20.479652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:20.540348 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:20.553176 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cf3b9-5d5f-4f72-9bca-024d4822622e None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:20.561229 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd269e-81ae-487a-88fa-404760235601 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:20.598838 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7563a4-9048-428a-a4c9-44493707fa58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:20.599580 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7563a4-9048-428a-a4c9-44493707fa58 None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:20.599907 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7563a4-9048-428a-a4c9-44493707fa58 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:20.600118 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7563a4-9048-428a-a4c9-44493707fa58 None admin] PATH_INFO: `/v3/groups` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:20.600753 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7563a4-9048-428a-a4c9-44493707fa58 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:20.601025 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103eac00-6cfe-4041-9adf-25e7eb3b3670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:20.601667 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103eac00-6cfe-4041-9adf-25e7eb3b3670 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:20.601947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103eac00-6cfe-4041-9adf-25e7eb3b3670 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:20.602199 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7563a4-9048-428a-a4c9-44493707fa58 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:20.602294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103eac00-6cfe-4041-9adf-25e7eb3b3670 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:20.613030 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103eac00-6cfe-4041-9adf-25e7eb3b3670 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:20.618412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-103eac00-6cfe-4041-9adf-25e7eb3b3670 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:07:20.618412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:20.618412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:20.618412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:20.618412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:20.618412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:20.618412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:20.618412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:20.618412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:20.618412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:20.618412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:20.618412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:20.618412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:20.618412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:20.618412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:20.618412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:20.619395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:20.619395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:20.619395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:20.619395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:20.619395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:20.619395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:20.619395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:20.619395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:20.619395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:20.619395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:20.619395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:20.619395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:20.619395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:20.619395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:20.619395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:20.619395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:20.620484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:20.620484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:20.620484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:20.620484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:20.620484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:20.620484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:20.620484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:20.620484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:20.620484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:20.620484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:20.620484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:07:20.620484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:20.621394 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 161/309] 10.222.0.67 () {60 vars in 1134 bytes} [Wed Aug 31 00:07:20 2022] POST /identity/v3/groups => generated 229 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:07:20.621466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 148/309] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:20.621530 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7563a4-9048-428a-a4c9-44493707fa58 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:20.632587 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103eac00-6cfe-4041-9adf-25e7eb3b3670 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:20.776901 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7563a4-9048-428a-a4c9-44493707fa58 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:20.783585 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103eac00-6cfe-4041-9adf-25e7eb3b3670 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:20.786089 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:20.825900 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a076c54-0cab-49b7-af75-1f2e7ba74b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:20.826506 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a076c54-0cab-49b7-af75-1f2e7ba74b04 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:20.826788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a076c54-0cab-49b7-af75-1f2e7ba74b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:20.827120 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a076c54-0cab-49b7-af75-1f2e7ba74b04 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:20.830719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095dc0c-d322-40a1-8b93-4eacf537a781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:20.832326 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095dc0c-d322-40a1-8b93-4eacf537a781 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:20.832326 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095dc0c-d322-40a1-8b93-4eacf537a781 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:20.832326 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095dc0c-d322-40a1-8b93-4eacf537a781 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:20.832818 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0095dc0c-d322-40a1-8b93-4eacf537a781 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:20.833119 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0095dc0c-d322-40a1-8b93-4eacf537a781 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:20.835053 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0095dc0c-d322-40a1-8b93-4eacf537a781 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:20.844445 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a076c54-0cab-49b7-af75-1f2e7ba74b04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:20.848892 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 149/310] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:07:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:20.854197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-4a076c54-0cab-49b7-af75-1f2e7ba74b04 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:07:20.854197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:20.854197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:20.854197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:20.854197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:20.854197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:20.854197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:20.854197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:20.854197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:20.854197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:20.854197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:20.854197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:20.854197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:20.854197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:20.854197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:20.854197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:20.855442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:20.855442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:20.855442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:20.855442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:20.855442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:20.855442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:20.855442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:20.855442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:20.855442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:20.855442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:20.855442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:20.855442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:20.855442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:20.855442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:20.855442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:20.855442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:20.856243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:20.856243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:20.856243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:20.856243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:20.856243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:20.856243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:20.856243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:20.856243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:20.856243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:20.856243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:20.856243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:07:20.856243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:20.856243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 162/311] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:20.859193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095dc0c-d322-40a1-8b93-4eacf537a781 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:20.861750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a076c54-0cab-49b7-af75-1f2e7ba74b04 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:20.875960 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:20.981131 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a076c54-0cab-49b7-af75-1f2e7ba74b04 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:20.983394 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095dc0c-d322-40a1-8b93-4eacf537a781 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:21.019138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7f97c-bd9d-4668-8f5d-4f2c51cfdab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:21.019815 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7f97c-bd9d-4668-8f5d-4f2c51cfdab4 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:21.020100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7f97c-bd9d-4668-8f5d-4f2c51cfdab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:21.020374 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7f97c-bd9d-4668-8f5d-4f2c51cfdab4 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:21.021046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7f97c-bd9d-4668-8f5d-4f2c51cfdab4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:21.021353 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7f97c-bd9d-4668-8f5d-4f2c51cfdab4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:21.022742 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7f97c-bd9d-4668-8f5d-4f2c51cfdab4 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:21.024621 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7b8ad-1a02-4bef-8900-0d973f1f3eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:21.025534 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7b8ad-1a02-4bef-8900-0d973f1f3eca None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:21.026017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7b8ad-1a02-4bef-8900-0d973f1f3eca None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:21.026476 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7b8ad-1a02-4bef-8900-0d973f1f3eca None admin] PATH_INFO: `/v3/users/8ecd95738567498382615ba7e2708836` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:21.031614 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 163/312] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:07:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:21.037299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7f97c-bd9d-4668-8f5d-4f2c51cfdab4 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:21.042018 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7b8ad-1a02-4bef-8900-0d973f1f3eca None admin] RBAC: Authorizing `identity:get_user(user_id=8ecd95738567498382615ba7e2708836)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:21.043510 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7b8ad-1a02-4bef-8900-0d973f1f3eca None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:21.050439 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 150/313] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:07:20 2022] GET /identity/v3/users/8ecd95738567498382615ba7e2708836 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:21.051718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7b8ad-1a02-4bef-8900-0d973f1f3eca None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:21.081917 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:21.136678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:21.171272 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7f97c-bd9d-4668-8f5d-4f2c51cfdab4 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:21.187902 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7b8ad-1a02-4bef-8900-0d973f1f3eca None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:21.212846 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb5c462-f0d7-41cf-a492-9e996a4e51a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:21.214279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5c462-f0d7-41cf-a492-9e996a4e51a9 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:21.214896 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5c462-f0d7-41cf-a492-9e996a4e51a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:21.215500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5c462-f0d7-41cf-a492-9e996a4e51a9 None admin] PATH_INFO: `/v3/users/c9fcdd29f7d741a78e3fc08852ad17a3` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:21.217507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb5c462-f0d7-41cf-a492-9e996a4e51a9 None admin] RBAC: Authorizing `identity:get_user(user_id=c9fcdd29f7d741a78e3fc08852ad17a3)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:21.219633 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb5c462-f0d7-41cf-a492-9e996a4e51a9 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:21.222432 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 164/314] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:07:21 2022] GET /identity/v3/users/c9fcdd29f7d741a78e3fc08852ad17a3 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:21.224542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e9e32-8b47-44f3-8b74-e6b0faa44e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:21.225148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e9e32-8b47-44f3-8b74-e6b0faa44e97 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:21.225423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e9e32-8b47-44f3-8b74-e6b0faa44e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:21.225694 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e9e32-8b47-44f3-8b74-e6b0faa44e97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:21.226422 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e9e32-8b47-44f3-8b74-e6b0faa44e97 None admin] RBAC: Adding query filter params (role.id=65d36a76676c424db12c090f01595bdc, user.id=8ecd95738567498382615ba7e2708836, scope.system=all) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:21.226794 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e9e32-8b47-44f3-8b74-e6b0faa44e97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:21.227815 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb5c462-f0d7-41cf-a492-9e996a4e51a9 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:21.228254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e9e32-8b47-44f3-8b74-e6b0faa44e97 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:21.234238 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 151/315] 10.222.0.67 () {56 vars in 1308 bytes} [Wed Aug 31 00:07:21 2022] GET /identity/v3/role_assignments?user.id=8ecd95738567498382615ba7e2708836&scope.system=all&role.id=65d36a76676c424db12c090f01595bdc => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:21.353482 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb5c462-f0d7-41cf-a492-9e996a4e51a9 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:21.393470 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06afc6-cf70-48ad-bfcb-fab3535816fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:21.394437 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06afc6-cf70-48ad-bfcb-fab3535816fa None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:21.394933 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06afc6-cf70-48ad-bfcb-fab3535816fa None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:21.395254 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06afc6-cf70-48ad-bfcb-fab3535816fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:21.421874 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:21.473237 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06afc6-cf70-48ad-bfcb-fab3535816fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:21.475019 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06afc6-cf70-48ad-bfcb-fab3535816fa None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:21.587850 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 165/316] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:21.594239 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e9e32-8b47-44f3-8b74-e6b0faa44e97 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:21.687613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:21.699627 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e9e32-8b47-44f3-8b74-e6b0faa44e97 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:21.742073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e2f95-644b-4536-a169-304da5f6ed21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:21.743081 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e2f95-644b-4536-a169-304da5f6ed21 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:21.743555 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e2f95-644b-4536-a169-304da5f6ed21 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:21.744052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e2f95-644b-4536-a169-304da5f6ed21 None admin] PATH_INFO: `/v3/projects/a8b13074ade34e4d944896277be959f5` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:21.745371 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e2f95-644b-4536-a169-304da5f6ed21 None admin] RBAC: Authorizing `identity:get_project(project_id=a8b13074ade34e4d944896277be959f5)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:21.747004 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e2f95-644b-4536-a169-304da5f6ed21 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:21.750326 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 152/317] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:21 2022] GET /identity/v3/projects/a8b13074ade34e4d944896277be959f5 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:21.755898 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06afc6-cf70-48ad-bfcb-fab3535816fa None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:21.802380 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:21.865930 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06afc6-cf70-48ad-bfcb-fab3535816fa None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:21.902536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a4b8c-0ae3-4363-a62c-8d3fbc8102ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:21.903176 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a4b8c-0ae3-4363-a62c-8d3fbc8102ce None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:21.903488 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a4b8c-0ae3-4363-a62c-8d3fbc8102ce None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:21.904330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a4b8c-0ae3-4363-a62c-8d3fbc8102ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:21.905029 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a4b8c-0ae3-4363-a62c-8d3fbc8102ce None admin] RBAC: Adding query filter params (role.id=65d36a76676c424db12c090f01595bdc, scope.project.id=a8b13074ade34e4d944896277be959f5, user.id=c9fcdd29f7d741a78e3fc08852ad17a3) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:21.905342 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a4b8c-0ae3-4363-a62c-8d3fbc8102ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:21.906856 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a4b8c-0ae3-4363-a62c-8d3fbc8102ce None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:21.913182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 166/318] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:07:21 2022] GET /identity/v3/role_assignments?user.id=c9fcdd29f7d741a78e3fc08852ad17a3&scope.project.id=a8b13074ade34e4d944896277be959f5&role.id=65d36a76676c424db12c090f01595bdc => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:22.490387 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff21ea-29b2-4fd6-9f48-5bdeff22f73a None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:22.490901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff21ea-29b2-4fd6-9f48-5bdeff22f73a None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:22.490978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff21ea-29b2-4fd6-9f48-5bdeff22f73a None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:22.492427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 153/319] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:22.497808 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3195fea-de62-49ef-be29-5b919a1df70b None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:22.498165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3195fea-de62-49ef-be29-5b919a1df70b None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:22.498438 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3195fea-de62-49ef-be29-5b919a1df70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:22.517371 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-b3195fea-de62-49ef-be29-5b919a1df70b None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:22.518541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b3195fea-de62-49ef-be29-5b919a1df70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:22.587420 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 167/320] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:22.702492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede3451-50ec-4399-9ec4-84bfa5b77bf8 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:22.702840 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede3451-50ec-4399-9ec4-84bfa5b77bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:22.703049 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede3451-50ec-4399-9ec4-84bfa5b77bf8 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:22.704549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 154/321] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:22.709761 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1490176-1763-4694-b6c5-b5db442325c6 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:22.710057 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1490176-1763-4694-b6c5-b5db442325c6 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:22.710334 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1490176-1763-4694-b6c5-b5db442325c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:22.728637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-f1490176-1763-4694-b6c5-b5db442325c6 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:22.729818 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f1490176-1763-4694-b6c5-b5db442325c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:22.757369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:22.800230 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 168/322] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:07:22.852530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede3451-50ec-4399-9ec4-84bfa5b77bf8 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:22.956422 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede3451-50ec-4399-9ec4-84bfa5b77bf8 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:22.995239 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a320a1-ed9d-4af3-8e5f-1cba19721a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:22.995880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a320a1-ed9d-4af3-8e5f-1cba19721a9b None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:22.996160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a320a1-ed9d-4af3-8e5f-1cba19721a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:22.996433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a320a1-ed9d-4af3-8e5f-1cba19721a9b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:23.007311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a320a1-ed9d-4af3-8e5f-1cba19721a9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:23.012572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-78a320a1-ed9d-4af3-8e5f-1cba19721a9b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:23.012572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:23.012572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:23.012572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:23.012572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:23.012572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:23.012572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:23.012572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:23.012572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:23.012572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:23.012572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:23.012572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:23.012572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:23.012572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:23.012572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:23.012572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:23.013530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:23.013530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:23.013530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:23.013530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:23.013530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:23.013530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:23.013530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:23.013530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:23.013530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:23.013530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:23.013530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:23.013530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:23.013530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:23.013530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:23.013530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:23.013530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:23.015821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:23.015821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:23.015821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:23.015821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:23.015821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:23.015821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:23.015821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:23.015821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:23.015821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:23.015821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:23.015821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:23.015821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:23.015821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 155/323] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:23.019524 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1490176-1763-4694-b6c5-b5db442325c6 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:23.078374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a320a1-ed9d-4af3-8e5f-1cba19721a9b None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:23.098437 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:23.129965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1490176-1763-4694-b6c5-b5db442325c6 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:23.170717 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef3806-371b-405b-ad39-54a69e9082c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:23.171383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef3806-371b-405b-ad39-54a69e9082c7 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:23.171664 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef3806-371b-405b-ad39-54a69e9082c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:23.171939 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef3806-371b-405b-ad39-54a69e9082c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:23.172602 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef3806-371b-405b-ad39-54a69e9082c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:23.172911 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef3806-371b-405b-ad39-54a69e9082c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:23.174297 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef3806-371b-405b-ad39-54a69e9082c7 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:23.187211 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 169/324] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:07:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:23.188554 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335eca0-12e9-416a-976b-99c3532b8c87 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:23.188843 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335eca0-12e9-416a-976b-99c3532b8c87 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:23.189122 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335eca0-12e9-416a-976b-99c3532b8c87 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:23.190207 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a320a1-ed9d-4af3-8e5f-1cba19721a9b None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:23.190398 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 170/325] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:23.191125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7335eca0-12e9-416a-976b-99c3532b8c87 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:23.220479 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:23.225533 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c380ab1-d6e6-4010-8e49-db7739043742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:23.226161 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c380ab1-d6e6-4010-8e49-db7739043742 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:23.226442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c380ab1-d6e6-4010-8e49-db7739043742 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:23.226711 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c380ab1-d6e6-4010-8e49-db7739043742 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:23.227849 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c380ab1-d6e6-4010-8e49-db7739043742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:23.229301 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c380ab1-d6e6-4010-8e49-db7739043742 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:23.230863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 156/326] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:23.232678 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b984d-9a2e-4284-8528-35f3495dfee4 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:23.232962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b984d-9a2e-4284-8528-35f3495dfee4 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:23.233237 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b984d-9a2e-4284-8528-35f3495dfee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:23.250844 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-160b984d-9a2e-4284-8528-35f3495dfee4 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:23.252048 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-160b984d-9a2e-4284-8528-35f3495dfee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:23.302430 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7335eca0-12e9-416a-976b-99c3532b8c87 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:23.322166 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 157/327] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:23.323105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b984d-9a2e-4284-8528-35f3495dfee4 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:23.339169 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:23.342059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34523c90-bebf-4a79-a96d-7df5b7dfcd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:23.342735 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34523c90-bebf-4a79-a96d-7df5b7dfcd23 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:23.343020 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34523c90-bebf-4a79-a96d-7df5b7dfcd23 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:23.343321 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34523c90-bebf-4a79-a96d-7df5b7dfcd23 None admin] PATH_INFO: `/v3/users/996eaf17b7f04faa905120a047d7f5b2` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:23.344546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34523c90-bebf-4a79-a96d-7df5b7dfcd23 None admin] RBAC: Authorizing `identity:get_user(user_id=996eaf17b7f04faa905120a047d7f5b2)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:23.346012 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34523c90-bebf-4a79-a96d-7df5b7dfcd23 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:23.347672 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 171/328] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:07:23 2022] GET /identity/v3/users/996eaf17b7f04faa905120a047d7f5b2 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:23.353397 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34523c90-bebf-4a79-a96d-7df5b7dfcd23 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:23.432794 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b984d-9a2e-4284-8528-35f3495dfee4 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:23.451501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:23.458958 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34523c90-bebf-4a79-a96d-7df5b7dfcd23 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:23.469354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa4f82-2b5e-4829-b995-bdf56a509cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:23.470152 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa4f82-2b5e-4829-b995-bdf56a509cf7 None admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:23.470428 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa4f82-2b5e-4829-b995-bdf56a509cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:23.470698 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa4f82-2b5e-4829-b995-bdf56a509cf7 None admin] PATH_INFO: `/v3/groups` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:23.471352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa4f82-2b5e-4829-b995-bdf56a509cf7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:23.472828 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa4f82-2b5e-4829-b995-bdf56a509cf7 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:23.490801 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 158/329] 10.222.0.67 () {60 vars in 1134 bytes} [Wed Aug 31 00:07:23 2022] POST /identity/v3/groups => generated 226 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:07:23.506708 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec625424-b38a-4144-bb3c-492316bb12a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:23.507385 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec625424-b38a-4144-bb3c-492316bb12a7 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:23.507667 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec625424-b38a-4144-bb3c-492316bb12a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:23.507937 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec625424-b38a-4144-bb3c-492316bb12a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:23.508689 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec625424-b38a-4144-bb3c-492316bb12a7 None admin] RBAC: Adding query filter params (role.id=cc8bd97c7b9745d1a52746650a2e49b9, user.id=996eaf17b7f04faa905120a047d7f5b2, scope.system=all) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:23.509074 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec625424-b38a-4144-bb3c-492316bb12a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:23.510575 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec625424-b38a-4144-bb3c-492316bb12a7 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:23.525107 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 172/330] 10.222.0.67 () {56 vars in 1308 bytes} [Wed Aug 31 00:07:23 2022] GET /identity/v3/role_assignments?user.id=996eaf17b7f04faa905120a047d7f5b2&scope.system=all&role.id=cc8bd97c7b9745d1a52746650a2e49b9 => generated 510 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:23.592125 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa4f82-2b5e-4829-b995-bdf56a509cf7 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:23.680964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:23.709040 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa4f82-2b5e-4829-b995-bdf56a509cf7 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:23.743842 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed544e-605f-41cb-a169-4b2a173df91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:23.744474 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed544e-605f-41cb-a169-4b2a173df91b None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:23.744752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed544e-605f-41cb-a169-4b2a173df91b None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:23.745021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed544e-605f-41cb-a169-4b2a173df91b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:23.756156 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed544e-605f-41cb-a169-4b2a173df91b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:23.761492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-1aed544e-605f-41cb-a169-4b2a173df91b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:07:23.761492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:23.761492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:23.761492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:23.761492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:23.761492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:23.761492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:23.761492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:23.761492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:23.761492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:23.761492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:23.761492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:23.761492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:23.761492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:23.761492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:23.761492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:23.762370 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:23.762370 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:23.762370 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:23.762370 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:23.762370 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:23.762370 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:23.762370 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:23.762370 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:23.762370 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:23.762370 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:23.762370 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:23.762370 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:23.762370 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:23.762370 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:23.762370 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:23.762370 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:23.763744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:23.763744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:23.763744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:23.763744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:23.763744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:23.763744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:23.763744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:23.763744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:23.763744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:23.763744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:23.763744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:07:23.763744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:23.763744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 159/331] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:23.769122 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec625424-b38a-4144-bb3c-492316bb12a7 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:23.882549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec625424-b38a-4144-bb3c-492316bb12a7 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:23.923121 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af42f5-f0e5-4249-a122-5a278e26cc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:23.923763 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af42f5-f0e5-4249-a122-5a278e26cc13 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:23.924047 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af42f5-f0e5-4249-a122-5a278e26cc13 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:23.924325 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af42f5-f0e5-4249-a122-5a278e26cc13 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:23.925006 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af42f5-f0e5-4249-a122-5a278e26cc13 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:23.925313 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af42f5-f0e5-4249-a122-5a278e26cc13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:23.926749 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af42f5-f0e5-4249-a122-5a278e26cc13 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:23.936718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 173/332] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:07:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:23.941956 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed544e-605f-41cb-a169-4b2a173df91b None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:24.057082 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed544e-605f-41cb-a169-4b2a173df91b None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:24.095237 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6950fa-e53f-41b4-bd5a-5fe94e775e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:24.096140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6950fa-e53f-41b4-bd5a-5fe94e775e1f None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:24.096556 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6950fa-e53f-41b4-bd5a-5fe94e775e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:24.097030 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6950fa-e53f-41b4-bd5a-5fe94e775e1f None admin] PATH_INFO: `/v3/users/8ecd95738567498382615ba7e2708836` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:24.099790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6950fa-e53f-41b4-bd5a-5fe94e775e1f None admin] RBAC: Authorizing `identity:get_user(user_id=8ecd95738567498382615ba7e2708836)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:24.101279 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6950fa-e53f-41b4-bd5a-5fe94e775e1f None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:24.102951 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 160/333] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:07:23 2022] GET /identity/v3/users/8ecd95738567498382615ba7e2708836 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:24.109454 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af42f5-f0e5-4249-a122-5a278e26cc13 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:24.133079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:24.233511 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af42f5-f0e5-4249-a122-5a278e26cc13 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:24.271359 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b16a5e-8d4f-4637-a326-bc00e877c9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:24.272437 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b16a5e-8d4f-4637-a326-bc00e877c9d4 None admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:24.272932 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b16a5e-8d4f-4637-a326-bc00e877c9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:24.273444 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b16a5e-8d4f-4637-a326-bc00e877c9d4 None admin] PATH_INFO: `/v3/system/users/8ecd95738567498382615ba7e2708836/roles/65d36a76676c424db12c090f01595bdc` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:24.275169 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b16a5e-8d4f-4637-a326-bc00e877c9d4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8ecd95738567498382615ba7e2708836, role_id=65d36a76676c424db12c090f01595bdc)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:24.276747 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b16a5e-8d4f-4637-a326-bc00e877c9d4 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:24.298808 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 174/334] 10.222.0.67 () {58 vars in 1334 bytes} [Wed Aug 31 00:07:24 2022] PUT /identity/v3/system/users/8ecd95738567498382615ba7e2708836/roles/65d36a76676c424db12c090f01595bdc => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:07:26.013499 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abc00d-7378-4379-b61a-c0da276fb24b None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:26.014573 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abc00d-7378-4379-b61a-c0da276fb24b None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:26.015111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abc00d-7378-4379-b61a-c0da276fb24b None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:26.017083 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 161/335] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:26.023289 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7fe2d-d333-4045-a355-b0bf51f28a49 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:26.024221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7fe2d-d333-4045-a355-b0bf51f28a49 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:26.024698 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7fe2d-d333-4045-a355-b0bf51f28a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:26.047946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-b8f7fe2d-d333-4045-a355-b0bf51f28a49 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:26.049488 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b8f7fe2d-d333-4045-a355-b0bf51f28a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:26.137639 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 175/336] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:26.387404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36714ee-5e38-455d-900a-6e0d72350243 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:26.388719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36714ee-5e38-455d-900a-6e0d72350243 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:26.389396 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36714ee-5e38-455d-900a-6e0d72350243 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:26.391027 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 162/337] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:26.396407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd344522-f68a-49d7-b11e-ab37d4ae4f7d None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:26.396932 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd344522-f68a-49d7-b11e-ab37d4ae4f7d None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:26.397437 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd344522-f68a-49d7-b11e-ab37d4ae4f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:26.420036 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-cd344522-f68a-49d7-b11e-ab37d4ae4f7d None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:26.421726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-cd344522-f68a-49d7-b11e-ab37d4ae4f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:26.424085 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:26.460976 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9c24f-2b67-49df-a72e-bcd6983dff0a None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:26.461288 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9c24f-2b67-49df-a72e-bcd6983dff0a None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:26.461527 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9c24f-2b67-49df-a72e-bcd6983dff0a None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:26.463147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 163/338] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:26.469985 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420540f-c5ee-4e8f-943c-e37238aac86d None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:26.470254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420540f-c5ee-4e8f-943c-e37238aac86d None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:26.470528 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420540f-c5ee-4e8f-943c-e37238aac86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:26.483402 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:26.504779 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-8420540f-c5ee-4e8f-943c-e37238aac86d None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:26.506337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-8420540f-c5ee-4e8f-943c-e37238aac86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:26.530471 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 176/339] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:26.610936 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 164/340] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:07:26.666659 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd344522-f68a-49d7-b11e-ab37d4ae4f7d None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:26.798975 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd344522-f68a-49d7-b11e-ab37d4ae4f7d None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:26.846595 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e1652-1f45-4049-ad56-dc57ce752cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:26.847561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e1652-1f45-4049-ad56-dc57ce752cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:26.847844 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e1652-1f45-4049-ad56-dc57ce752cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:26.848118 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e1652-1f45-4049-ad56-dc57ce752cc4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:26.860698 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:26.865473 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93e1652-1f45-4049-ad56-dc57ce752cc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:26.874086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-d93e1652-1f45-4049-ad56-dc57ce752cc4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:26.874086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:26.874086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:26.874086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:26.874086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:26.874086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:26.874086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:26.874086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:26.874086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:26.874086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:26.874086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:26.874086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:26.874086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:26.874086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:26.874086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:26.874086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:26.875292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:26.875292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:26.875292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:26.875292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:26.875292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:26.875292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:26.875292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:26.875292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:26.875292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:26.875292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:26.875292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:26.875292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:26.875292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:26.875292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:26.875292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:26.875292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:26.876100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:26.876100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:26.876100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:26.876100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:26.876100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:26.876100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:26.876100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:26.876100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:26.876100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:26.876100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:26.876100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:26.876100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:26.876100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 177/341] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:26.881689 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420540f-c5ee-4e8f-943c-e37238aac86d None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:26.941545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e1652-1f45-4049-ad56-dc57ce752cc4 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:27.074544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420540f-c5ee-4e8f-943c-e37238aac86d None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:27.097637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e1652-1f45-4049-ad56-dc57ce752cc4 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:27.118176 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:27.124825 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad672f-6ded-4db5-aeff-72db058473b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:27.126213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad672f-6ded-4db5-aeff-72db058473b0 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:27.126708 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad672f-6ded-4db5-aeff-72db058473b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:27.127209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad672f-6ded-4db5-aeff-72db058473b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:27.128072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad672f-6ded-4db5-aeff-72db058473b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:27.128567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad672f-6ded-4db5-aeff-72db058473b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:27.130153 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad672f-6ded-4db5-aeff-72db058473b0 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:27.140356 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 165/342] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:07:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:27.142133 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5f373-15db-49b8-972f-ede8f28eeb0d None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:27.142610 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5f373-15db-49b8-972f-ede8f28eeb0d None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:27.143101 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5f373-15db-49b8-972f-ede8f28eeb0d None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:27.144806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 166/343] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:27.145727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5f373-15db-49b8-972f-ede8f28eeb0d None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:27.152935 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd33ddc-6053-4418-8a4a-9ded411d6b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:27.154072 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd33ddc-6053-4418-8a4a-9ded411d6b68 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:27.154553 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd33ddc-6053-4418-8a4a-9ded411d6b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:27.155065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd33ddc-6053-4418-8a4a-9ded411d6b68 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:27.168307 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd33ddc-6053-4418-8a4a-9ded411d6b68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:27.175637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-ddd33ddc-6053-4418-8a4a-9ded411d6b68 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:07:27.175637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:27.175637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:27.175637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:27.175637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:27.175637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:27.175637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:27.175637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:27.175637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:27.175637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:27.175637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:27.175637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:27.175637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:27.175637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:27.175637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:27.175637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:27.176536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:27.176536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:27.176536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:27.176536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:27.176536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:27.176536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:27.176536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:27.176536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:27.176536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:27.176536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:27.176536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:27.176536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:27.176536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:27.176536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:27.176536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:27.176536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:27.177308 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:27.177308 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:27.177308 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:27.177308 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:27.177308 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:27.177308 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:27.177308 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:27.177308 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:27.177308 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:27.177308 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:27.177308 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:07:27.177308 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:27.180107 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 178/344] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 31 00:07:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:07:27.183073 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd33ddc-6053-4418-8a4a-9ded411d6b68 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:27.270257 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:27.287625 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5f373-15db-49b8-972f-ede8f28eeb0d None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:27.330106 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd33ddc-6053-4418-8a4a-9ded411d6b68 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:27.336451 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea607db5-20a5-4460-a9c5-270a928ba0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:27.337385 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea607db5-20a5-4460-a9c5-270a928ba0b8 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:27.337841 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea607db5-20a5-4460-a9c5-270a928ba0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:27.338262 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea607db5-20a5-4460-a9c5-270a928ba0b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:27.351191 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea607db5-20a5-4460-a9c5-270a928ba0b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:27.359075 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-ea607db5-20a5-4460-a9c5-270a928ba0b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:27.359075 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:27.359075 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:27.359075 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:27.359075 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:27.359075 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:27.359075 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:27.359075 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:27.359075 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:27.359075 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:27.359075 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:27.359075 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:27.359075 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:27.359075 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:27.359075 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:27.359075 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:27.360209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:27.360209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:27.360209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:27.360209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:27.360209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:27.360209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:27.360209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:27.360209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:27.360209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:27.360209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:27.360209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:27.360209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:27.360209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:27.360209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:27.360209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:27.360209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:27.361198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:27.361198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:27.361198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:27.361198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:27.361198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:27.361198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:27.361198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:27.361198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:27.361198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:27.361198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:27.361198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:27.361198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:27.363341 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 167/345] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:27.364700 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3289d-292d-47e4-970a-1dfce81512d0 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:27.365156 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3289d-292d-47e4-970a-1dfce81512d0 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:27.365653 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3289d-292d-47e4-970a-1dfce81512d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:27.371372 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af6652-21e6-4368-a6ab-01af0e9baccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:27.371989 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af6652-21e6-4368-a6ab-01af0e9baccf None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:27.372446 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af6652-21e6-4368-a6ab-01af0e9baccf None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:27.372831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af6652-21e6-4368-a6ab-01af0e9baccf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:27.390383 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-32f3289d-292d-47e4-970a-1dfce81512d0 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:27.391998 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-32f3289d-292d-47e4-970a-1dfce81512d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:27.449360 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:27.453122 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af6652-21e6-4368-a6ab-01af0e9baccf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:27.454714 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af6652-21e6-4368-a6ab-01af0e9baccf None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:27.475571 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 168/346] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:27.476749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3289d-292d-47e4-970a-1dfce81512d0 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:27.520773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:27.580507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 179/347] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:27.581375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af6652-21e6-4368-a6ab-01af0e9baccf None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:27.633596 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3289d-292d-47e4-970a-1dfce81512d0 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:27.677052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb708db2-3ab9-4a47-a9e6-e1adce93c563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:27.678028 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb708db2-3ab9-4a47-a9e6-e1adce93c563 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:27.678504 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb708db2-3ab9-4a47-a9e6-e1adce93c563 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:27.678974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb708db2-3ab9-4a47-a9e6-e1adce93c563 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:27.679874 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb708db2-3ab9-4a47-a9e6-e1adce93c563 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:27.680399 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb708db2-3ab9-4a47-a9e6-e1adce93c563 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:27.681972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb708db2-3ab9-4a47-a9e6-e1adce93c563 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:27.691699 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 169/348] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:07:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:27.693317 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1555d58-48a6-4ada-b3ce-1acae045966c None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:27.693803 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1555d58-48a6-4ada-b3ce-1acae045966c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:27.694308 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1555d58-48a6-4ada-b3ce-1acae045966c None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:27.696071 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 170/349] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:27.696977 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1555d58-48a6-4ada-b3ce-1acae045966c None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:27.707707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af6652-21e6-4368-a6ab-01af0e9baccf None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:27.746062 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c52544-7097-40bf-b864-63d0c7e3b6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:27.748088 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c52544-7097-40bf-b864-63d0c7e3b6bd None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:27.748088 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c52544-7097-40bf-b864-63d0c7e3b6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:27.748088 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c52544-7097-40bf-b864-63d0c7e3b6bd None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:27.748443 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c52544-7097-40bf-b864-63d0c7e3b6bd None admin] RBAC: Adding query filter params (name=member) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:27.748780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c52544-7097-40bf-b864-63d0c7e3b6bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:27.750559 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c52544-7097-40bf-b864-63d0c7e3b6bd None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:27.757106 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:27.762292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 180/350] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:07:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:27.764210 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b04e77-3d2b-47f9-8019-9e253174f064 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:27.764880 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b04e77-3d2b-47f9-8019-9e253174f064 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:27.765164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b04e77-3d2b-47f9-8019-9e253174f064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:27.786236 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-47b04e77-3d2b-47f9-8019-9e253174f064 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:27.787294 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-47b04e77-3d2b-47f9-8019-9e253174f064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:27.804118 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:27.820534 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1555d58-48a6-4ada-b3ce-1acae045966c None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:27.857017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec1976-3538-4289-9881-7d6db3d8fc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:27.858165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec1976-3538-4289-9881-7d6db3d8fc04 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:27.858644 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec1976-3538-4289-9881-7d6db3d8fc04 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:27.859165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec1976-3538-4289-9881-7d6db3d8fc04 None admin] PATH_INFO: `/v3/projects/9eb64431e703423581c08ff8f8378eb3` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:27.860603 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec1976-3538-4289-9881-7d6db3d8fc04 None admin] RBAC: Authorizing `identity:get_project(project_id=9eb64431e703423581c08ff8f8378eb3)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:27.862062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec1976-3538-4289-9881-7d6db3d8fc04 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:27.864833 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 171/351] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:27 2022] GET /identity/v3/projects/9eb64431e703423581c08ff8f8378eb3 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:27.864833 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec1976-3538-4289-9881-7d6db3d8fc04 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:27.875271 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 181/352] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:27.875271 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b04e77-3d2b-47f9-8019-9e253174f064 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:27.990060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec1976-3538-4289-9881-7d6db3d8fc04 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:28.008417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:28.034406 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd02cb-459e-4cd7-b943-3ce1f94b235a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:28.035094 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd02cb-459e-4cd7-b943-3ce1f94b235a None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:28.035552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd02cb-459e-4cd7-b943-3ce1f94b235a None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:28.035678 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd02cb-459e-4cd7-b943-3ce1f94b235a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:28.046226 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b04e77-3d2b-47f9-8019-9e253174f064 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:28.095871 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710ce4a-efc5-47de-ba72-101116318916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:28.096854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710ce4a-efc5-47de-ba72-101116318916 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:28.097382 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710ce4a-efc5-47de-ba72-101116318916 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:28.097863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710ce4a-efc5-47de-ba72-101116318916 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:28.116728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd02cb-459e-4cd7-b943-3ce1f94b235a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:28.118537 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd02cb-459e-4cd7-b943-3ce1f94b235a None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:28.130534 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:28.199621 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9710ce4a-efc5-47de-ba72-101116318916 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:28.201612 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9710ce4a-efc5-47de-ba72-101116318916 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:28.251078 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 172/353] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:28.251936 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd02cb-459e-4cd7-b943-3ce1f94b235a None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:28.272956 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:28.326179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 182/354] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:28.327163 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710ce4a-efc5-47de-ba72-101116318916 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:28.386269 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd02cb-459e-4cd7-b943-3ce1f94b235a None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:28.386722 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:28.430225 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c85bcd-fa9b-41f8-9e95-c6b0cd033c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:28.430855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c85bcd-fa9b-41f8-9e95-c6b0cd033c72 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:28.431168 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c85bcd-fa9b-41f8-9e95-c6b0cd033c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:28.431443 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c85bcd-fa9b-41f8-9e95-c6b0cd033c72 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:28.444814 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c85bcd-fa9b-41f8-9e95-c6b0cd033c72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:28.446075 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710ce4a-efc5-47de-ba72-101116318916 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:28.449826 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-d2c85bcd-fa9b-41f8-9e95-c6b0cd033c72 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:07:28.449826 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:28.449826 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:28.449826 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:28.449826 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:28.449826 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:28.449826 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:28.449826 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:28.449826 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:28.449826 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:28.449826 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:28.449826 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:28.449826 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:28.449826 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:28.449826 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:28.449826 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:28.451780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:28.451780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:28.451780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:28.451780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:28.451780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:28.451780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:28.451780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:28.451780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:28.451780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:28.451780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:28.451780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:28.451780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:28.451780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:28.451780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:28.451780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:28.451780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:28.452579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:28.452579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:28.452579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:28.452579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:28.452579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:28.452579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:28.452579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:28.452579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:28.452579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:28.452579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:28.452579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:07:28.452579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:28.456786 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 173/355] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 31 00:07:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:28.457652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c85bcd-fa9b-41f8-9e95-c6b0cd033c72 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:28.485541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c90e8-f361-4fc9-8b3d-7d923e9a5ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:28.486186 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c90e8-f361-4fc9-8b3d-7d923e9a5ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:28.486468 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c90e8-f361-4fc9-8b3d-7d923e9a5ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:28.486738 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c90e8-f361-4fc9-8b3d-7d923e9a5ae9 None admin] PATH_INFO: `/v3/groups/dd13637587354a10893278b6d62d73ca` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:28.496766 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c90e8-f361-4fc9-8b3d-7d923e9a5ae9 None admin] RBAC: Authorizing `identity:get_group(group_id=dd13637587354a10893278b6d62d73ca)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:28.498797 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c90e8-f361-4fc9-8b3d-7d923e9a5ae9 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:28.500923 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 183/356] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 31 00:07:28 2022] GET /identity/v3/groups/dd13637587354a10893278b6d62d73ca => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:28.501917 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdee04b-8215-4337-881b-a15701a616a6 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:28.502205 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdee04b-8215-4337-881b-a15701a616a6 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:28.502483 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdee04b-8215-4337-881b-a15701a616a6 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:28.503722 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 184/357] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:28.504410 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdee04b-8215-4337-881b-a15701a616a6 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:28.551385 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:28.584752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c85bcd-fa9b-41f8-9e95-c6b0cd033c72 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:28.618005 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdee04b-8215-4337-881b-a15701a616a6 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:28.631521 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054986d-d593-4e39-ab45-8fd2373892b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:28.632165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054986d-d593-4e39-ab45-8fd2373892b4 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:28.632447 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054986d-d593-4e39-ab45-8fd2373892b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:28.632724 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054986d-d593-4e39-ab45-8fd2373892b4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:28.632991 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:28.644761 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1054986d-d593-4e39-ab45-8fd2373892b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:28.649633 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-1054986d-d593-4e39-ab45-8fd2373892b4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:07:28.649633 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:28.649633 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:28.649633 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:28.649633 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:28.649633 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:28.649633 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:28.649633 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:28.649633 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:28.649633 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:28.649633 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:28.649633 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:28.649633 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:28.649633 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:28.649633 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:28.649633 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:28.650670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:28.650670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:28.650670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:28.650670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:28.650670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:28.650670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:28.650670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:28.650670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:28.650670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:28.650670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:28.650670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:28.650670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:28.650670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:28.650670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:28.650670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:28.650670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:28.651505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:28.651505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:28.651505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:28.651505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:28.651505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:28.651505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:28.651505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:28.651505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:28.651505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:28.651505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:28.651505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 00:07:28.651505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:28.651505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 174/358] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:28.652182 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054986d-d593-4e39-ab45-8fd2373892b4 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:28.662942 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e040d7-7c64-46da-813f-4ab28ddcf3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:28.663762 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e040d7-7c64-46da-813f-4ab28ddcf3ee None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:28.664046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e040d7-7c64-46da-813f-4ab28ddcf3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:28.664323 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e040d7-7c64-46da-813f-4ab28ddcf3ee None admin] PATH_INFO: `/v3/projects/9eb64431e703423581c08ff8f8378eb3` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:28.665522 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e040d7-7c64-46da-813f-4ab28ddcf3ee None admin] RBAC: Authorizing `identity:get_project(project_id=9eb64431e703423581c08ff8f8378eb3)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:28.666977 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e040d7-7c64-46da-813f-4ab28ddcf3ee None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:28.668811 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 185/359] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:28 2022] GET /identity/v3/projects/9eb64431e703423581c08ff8f8378eb3 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:28.670270 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e040d7-7c64-46da-813f-4ab28ddcf3ee None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:28.772474 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054986d-d593-4e39-ab45-8fd2373892b4 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:28.795296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e040d7-7c64-46da-813f-4ab28ddcf3ee None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:28.817269 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a82437b-2939-4d45-864b-6fe2859bda73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:28.817912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a82437b-2939-4d45-864b-6fe2859bda73 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:28.818191 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a82437b-2939-4d45-864b-6fe2859bda73 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:28.818469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a82437b-2939-4d45-864b-6fe2859bda73 None admin] PATH_INFO: `/v3/projects/a8b13074ade34e4d944896277be959f5` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:28.819673 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a82437b-2939-4d45-864b-6fe2859bda73 None admin] RBAC: Authorizing `identity:get_project(project_id=a8b13074ade34e4d944896277be959f5)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:28.821114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a82437b-2939-4d45-864b-6fe2859bda73 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:28.822766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 175/360] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:28 2022] GET /identity/v3/projects/a8b13074ade34e4d944896277be959f5 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:28.823563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a82437b-2939-4d45-864b-6fe2859bda73 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:28.826200 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:28.847371 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc5bdbe-9302-4cbd-93fd-5e8009440e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:28.847929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5bdbe-9302-4cbd-93fd-5e8009440e75 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:28.848211 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5bdbe-9302-4cbd-93fd-5e8009440e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:28.848482 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5bdbe-9302-4cbd-93fd-5e8009440e75 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:28.849165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc5bdbe-9302-4cbd-93fd-5e8009440e75 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:28.849470 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc5bdbe-9302-4cbd-93fd-5e8009440e75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:28.852447 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc5bdbe-9302-4cbd-93fd-5e8009440e75 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:28.860563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 186/361] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:07:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:28.862328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d651e7f-f819-4e76-a4ba-c7addcede6c7 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:28.862328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d651e7f-f819-4e76-a4ba-c7addcede6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:28.862328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d651e7f-f819-4e76-a4ba-c7addcede6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:28.885516 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-0d651e7f-f819-4e76-a4ba-c7addcede6c7 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:28.886731 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-0d651e7f-f819-4e76-a4ba-c7addcede6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:28.936846 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:28.953688 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a82437b-2939-4d45-864b-6fe2859bda73 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:28.968728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 187/362] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:28.971373 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d651e7f-f819-4e76-a4ba-c7addcede6c7 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:28.994301 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df76f46-6f2a-4294-8b94-b90533336295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:28.994983 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df76f46-6f2a-4294-8b94-b90533336295 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:28.995369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df76f46-6f2a-4294-8b94-b90533336295 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:28.995655 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df76f46-6f2a-4294-8b94-b90533336295 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:28.996393 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df76f46-6f2a-4294-8b94-b90533336295 None admin] RBAC: Adding query filter params (group.id=dd13637587354a10893278b6d62d73ca, role.id=cc8bd97c7b9745d1a52746650a2e49b9, scope.project.id=9eb64431e703423581c08ff8f8378eb3) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:28.996706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df76f46-6f2a-4294-8b94-b90533336295 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:28.998137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df76f46-6f2a-4294-8b94-b90533336295 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:29.005389 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 176/363] 10.222.0.67 () {56 vars in 1376 bytes} [Wed Aug 31 00:07:28 2022] GET /identity/v3/role_assignments?group.id=dd13637587354a10893278b6d62d73ca&scope.project.id=9eb64431e703423581c08ff8f8378eb3&role.id=cc8bd97c7b9745d1a52746650a2e49b9 => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:29.005389 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df76f46-6f2a-4294-8b94-b90533336295 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:29.087327 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:29.113078 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d651e7f-f819-4e76-a4ba-c7addcede6c7 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:29.136199 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df76f46-6f2a-4294-8b94-b90533336295 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:29.170773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbee808-beb3-4095-b4fb-9a54ff6c5af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:29.173148 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbee808-beb3-4095-b4fb-9a54ff6c5af9 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:29.173779 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbee808-beb3-4095-b4fb-9a54ff6c5af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:29.174423 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbee808-beb3-4095-b4fb-9a54ff6c5af9 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:29.175962 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbee808-beb3-4095-b4fb-9a54ff6c5af9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:29.176219 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbee808-beb3-4095-b4fb-9a54ff6c5af9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:29.177658 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbee808-beb3-4095-b4fb-9a54ff6c5af9 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:29.190399 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 188/364] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:07:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:29.191844 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbee808-beb3-4095-b4fb-9a54ff6c5af9 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:29.192781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678a04b-cc07-4b1a-b68f-9e161cae9d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:29.193718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678a04b-cc07-4b1a-b68f-9e161cae9d01 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:29.193997 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678a04b-cc07-4b1a-b68f-9e161cae9d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:29.194272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678a04b-cc07-4b1a-b68f-9e161cae9d01 None admin] PATH_INFO: `/v3/groups/dd13637587354a10893278b6d62d73ca` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:29.195869 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9678a04b-cc07-4b1a-b68f-9e161cae9d01 None admin] RBAC: Authorizing `identity:get_group(group_id=dd13637587354a10893278b6d62d73ca)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:29.197310 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9678a04b-cc07-4b1a-b68f-9e161cae9d01 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:29.201951 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 177/365] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 31 00:07:29 2022] GET /identity/v3/groups/dd13637587354a10893278b6d62d73ca => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:29.203144 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678a04b-cc07-4b1a-b68f-9e161cae9d01 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:29.256256 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:29.333883 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbee808-beb3-4095-b4fb-9a54ff6c5af9 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:29.336827 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678a04b-cc07-4b1a-b68f-9e161cae9d01 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:29.376434 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd61b84-5d18-44db-b991-cc3d9cfebf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:29.376995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd61b84-5d18-44db-b991-cc3d9cfebf4c None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:29.377279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd61b84-5d18-44db-b991-cc3d9cfebf4c None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:29.377554 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd61b84-5d18-44db-b991-cc3d9cfebf4c None admin] PATH_INFO: `/v3/groups/dd13637587354a10893278b6d62d73ca` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:29.378726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd61b84-5d18-44db-b991-cc3d9cfebf4c None admin] RBAC: Authorizing `identity:get_group(group_id=dd13637587354a10893278b6d62d73ca)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:29.380176 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd61b84-5d18-44db-b991-cc3d9cfebf4c None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:29.381780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 189/366] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 31 00:07:29 2022] GET /identity/v3/groups/dd13637587354a10893278b6d62d73ca => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:29.382522 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd61b84-5d18-44db-b991-cc3d9cfebf4c None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:29.383386 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb940ebd-5bbc-40f1-870a-84fe46a30966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:29.384237 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb940ebd-5bbc-40f1-870a-84fe46a30966 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:29.384724 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb940ebd-5bbc-40f1-870a-84fe46a30966 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:29.385477 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb940ebd-5bbc-40f1-870a-84fe46a30966 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:29.395893 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:29.471990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb940ebd-5bbc-40f1-870a-84fe46a30966 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:29.473474 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb940ebd-5bbc-40f1-870a-84fe46a30966 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:29.524556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd61b84-5d18-44db-b991-cc3d9cfebf4c None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:29.531136 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:29.582035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f2f28-3269-445d-bdda-33646e7b0290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:29.582673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f2f28-3269-445d-bdda-33646e7b0290 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:29.582952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f2f28-3269-445d-bdda-33646e7b0290 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:29.583575 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f2f28-3269-445d-bdda-33646e7b0290 None admin] PATH_INFO: `/v3/users/8ecd95738567498382615ba7e2708836` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:29.584475 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f2f28-3269-445d-bdda-33646e7b0290 None admin] RBAC: Authorizing `identity:get_user(user_id=8ecd95738567498382615ba7e2708836)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:29.585911 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f2f28-3269-445d-bdda-33646e7b0290 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:29.587593 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 190/367] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:07:29 2022] GET /identity/v3/users/8ecd95738567498382615ba7e2708836 => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:29.593124 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f2f28-3269-445d-bdda-33646e7b0290 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:29.600821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 178/368] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:29.601580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb940ebd-5bbc-40f1-870a-84fe46a30966 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:29.652706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:29.710726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f2f28-3269-445d-bdda-33646e7b0290 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:29.741646 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb940ebd-5bbc-40f1-870a-84fe46a30966 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:29.753946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f84c80-038e-4c38-b135-6b680f370706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:29.754934 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f84c80-038e-4c38-b135-6b680f370706 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:29.755463 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f84c80-038e-4c38-b135-6b680f370706 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:29.755928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f84c80-038e-4c38-b135-6b680f370706 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:29.757286 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f84c80-038e-4c38-b135-6b680f370706 None admin] RBAC: Adding query filter params (group.id=dd13637587354a10893278b6d62d73ca, role.id=4d91c00f40304b9e9ef4e6c89f46e5c5, scope.project.id=9eb64431e703423581c08ff8f8378eb3) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:29.757814 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f84c80-038e-4c38-b135-6b680f370706 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:29.759509 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f84c80-038e-4c38-b135-6b680f370706 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:29.766345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 191/369] 10.222.0.67 () {56 vars in 1376 bytes} [Wed Aug 31 00:07:29 2022] GET /identity/v3/role_assignments?group.id=dd13637587354a10893278b6d62d73ca&scope.project.id=9eb64431e703423581c08ff8f8378eb3&role.id=4d91c00f40304b9e9ef4e6c89f46e5c5 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:29.767479 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f84c80-038e-4c38-b135-6b680f370706 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:29.787540 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b677d2-77a9-4761-ae45-07649bff0176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:29.788412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b677d2-77a9-4761-ae45-07649bff0176 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:29.788691 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b677d2-77a9-4761-ae45-07649bff0176 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:29.788963 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b677d2-77a9-4761-ae45-07649bff0176 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:29.804226 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b677d2-77a9-4761-ae45-07649bff0176 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:29.811491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-04b677d2-77a9-4761-ae45-07649bff0176 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:07:29.811491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:29.811491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:29.811491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:29.811491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:29.811491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:29.811491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:29.811491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:29.811491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:29.811491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:29.811491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:29.811491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:29.811491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:29.811491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:29.811491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:29.811491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:29.812489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:29.812489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:29.812489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:29.812489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:29.812489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:29.812489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:29.812489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:29.812489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:29.812489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:29.812489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:29.812489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:29.812489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:29.812489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:29.812489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:29.812489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:29.812489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:29.813259 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:29.813259 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:29.813259 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:29.813259 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:29.813259 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:29.813259 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:29.813259 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:29.813259 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:29.813259 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:29.813259 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:29.813259 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:07:29.813259 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:29.813259 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 179/370] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:07:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:29.813920 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b677d2-77a9-4761-ae45-07649bff0176 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:29.831463 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:29.922608 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f84c80-038e-4c38-b135-6b680f370706 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:29.949018 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:29.959245 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b677d2-77a9-4761-ae45-07649bff0176 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:29.968401 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5516f3b-1300-493d-b2bd-ccf19a389a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:29.969021 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5516f3b-1300-493d-b2bd-ccf19a389a91 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:29.969297 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5516f3b-1300-493d-b2bd-ccf19a389a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:29.969564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5516f3b-1300-493d-b2bd-ccf19a389a91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:29.970303 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5516f3b-1300-493d-b2bd-ccf19a389a91 None admin] RBAC: Adding query filter params (group.id=dd13637587354a10893278b6d62d73ca, role.id=cc8bd97c7b9745d1a52746650a2e49b9, scope.project.id=a8b13074ade34e4d944896277be959f5) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:29.970602 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5516f3b-1300-493d-b2bd-ccf19a389a91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:29.972184 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5516f3b-1300-493d-b2bd-ccf19a389a91 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:29.981398 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 192/371] 10.222.0.67 () {56 vars in 1376 bytes} [Wed Aug 31 00:07:29 2022] GET /identity/v3/role_assignments?group.id=dd13637587354a10893278b6d62d73ca&scope.project.id=a8b13074ade34e4d944896277be959f5&role.id=cc8bd97c7b9745d1a52746650a2e49b9 => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:29.988166 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5516f3b-1300-493d-b2bd-ccf19a389a91 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:30.052288 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45c5a4-df33-4a22-941a-a9e1b035d8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:30.052918 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45c5a4-df33-4a22-941a-a9e1b035d8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:30.053196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45c5a4-df33-4a22-941a-a9e1b035d8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:30.053465 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45c5a4-df33-4a22-941a-a9e1b035d8b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:30.054210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a45c5a4-df33-4a22-941a-a9e1b035d8b0 None admin] RBAC: Adding query filter params (role.id=65d36a76676c424db12c090f01595bdc, user.id=8ecd95738567498382615ba7e2708836, scope.system=all) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:30.054520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a45c5a4-df33-4a22-941a-a9e1b035d8b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:30.056131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a45c5a4-df33-4a22-941a-a9e1b035d8b0 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:30.085176 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 180/372] 10.222.0.67 () {56 vars in 1308 bytes} [Wed Aug 31 00:07:29 2022] GET /identity/v3/role_assignments?user.id=8ecd95738567498382615ba7e2708836&scope.system=all&role.id=65d36a76676c424db12c090f01595bdc => generated 510 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:30.087453 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45c5a4-df33-4a22-941a-a9e1b035d8b0 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:30.164753 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:30.173361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5516f3b-1300-493d-b2bd-ccf19a389a91 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:30.275065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ef6ec-e064-4902-8ada-9aab634de15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:30.276252 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ef6ec-e064-4902-8ada-9aab634de15f None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:30.277544 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ef6ec-e064-4902-8ada-9aab634de15f None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:30.281054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ef6ec-e064-4902-8ada-9aab634de15f None admin] PATH_INFO: `/v3/projects/a8b13074ade34e4d944896277be959f5` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:30.282694 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7ef6ec-e064-4902-8ada-9aab634de15f None admin] RBAC: Authorizing `identity:get_project(project_id=a8b13074ade34e4d944896277be959f5)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:30.284433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7ef6ec-e064-4902-8ada-9aab634de15f None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:30.287613 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 193/373] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:29 2022] GET /identity/v3/projects/a8b13074ade34e4d944896277be959f5 => generated 302 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:30.294661 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ef6ec-e064-4902-8ada-9aab634de15f None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:30.313789 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45c5a4-df33-4a22-941a-a9e1b035d8b0 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:30.371901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc2200-1666-46d3-aba4-e7249fe6da5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:30.373384 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc2200-1666-46d3-aba4-e7249fe6da5d None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:30.373766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc2200-1666-46d3-aba4-e7249fe6da5d None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:30.374045 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc2200-1666-46d3-aba4-e7249fe6da5d None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:30.374725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cc2200-1666-46d3-aba4-e7249fe6da5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:30.376537 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cc2200-1666-46d3-aba4-e7249fe6da5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:30.377939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cc2200-1666-46d3-aba4-e7249fe6da5d None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:30.391467 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 181/374] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:30.397628 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc2200-1666-46d3-aba4-e7249fe6da5d None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:30.426894 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:30.477017 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ef6ec-e064-4902-8ada-9aab634de15f None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:30.526599 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc2200-1666-46d3-aba4-e7249fe6da5d None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:30.551767 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f43b4-c8ad-4364-ad3c-dd996817f7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:30.552433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f43b4-c8ad-4364-ad3c-dd996817f7cd None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:30.552716 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f43b4-c8ad-4364-ad3c-dd996817f7cd None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:30.552996 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f43b4-c8ad-4364-ad3c-dd996817f7cd None admin] PATH_INFO: `/v3/groups/dd13637587354a10893278b6d62d73ca` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:30.554212 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7f43b4-c8ad-4364-ad3c-dd996817f7cd None admin] RBAC: Authorizing `identity:get_group(group_id=dd13637587354a10893278b6d62d73ca)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:30.555704 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7f43b4-c8ad-4364-ad3c-dd996817f7cd None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:30.562210 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 194/375] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 31 00:07:30 2022] GET /identity/v3/groups/dd13637587354a10893278b6d62d73ca => generated 226 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:30.564703 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f43b4-c8ad-4364-ad3c-dd996817f7cd None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:30.571748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156345c0-12e1-40ff-aaf9-7b4fcab14450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:30.572382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156345c0-12e1-40ff-aaf9-7b4fcab14450 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:30.572666 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156345c0-12e1-40ff-aaf9-7b4fcab14450 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:30.572936 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156345c0-12e1-40ff-aaf9-7b4fcab14450 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:30.611391 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:30.674231 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156345c0-12e1-40ff-aaf9-7b4fcab14450 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:30.676590 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156345c0-12e1-40ff-aaf9-7b4fcab14450 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:30.696530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:30.777469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f43b4-c8ad-4364-ad3c-dd996817f7cd None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:30.821996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 182/376] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:30.828107 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156345c0-12e1-40ff-aaf9-7b4fcab14450 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:30.852307 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea7b44b-2751-4bc9-9d02-0cf0e006f5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:30.852937 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7b44b-2751-4bc9-9d02-0cf0e006f5df None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:30.853215 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7b44b-2751-4bc9-9d02-0cf0e006f5df None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:30.854228 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7b44b-2751-4bc9-9d02-0cf0e006f5df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:30.854969 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea7b44b-2751-4bc9-9d02-0cf0e006f5df None admin] RBAC: Adding query filter params (group.id=dd13637587354a10893278b6d62d73ca, role.id=4d91c00f40304b9e9ef4e6c89f46e5c5, scope.project.id=a8b13074ade34e4d944896277be959f5) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:30.855314 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea7b44b-2751-4bc9-9d02-0cf0e006f5df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:30.857324 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea7b44b-2751-4bc9-9d02-0cf0e006f5df None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:30.875798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 195/377] 10.222.0.67 () {56 vars in 1376 bytes} [Wed Aug 31 00:07:30 2022] GET /identity/v3/role_assignments?group.id=dd13637587354a10893278b6d62d73ca&scope.project.id=a8b13074ade34e4d944896277be959f5&role.id=4d91c00f40304b9e9ef4e6c89f46e5c5 => generated 260 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:30.972344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156345c0-12e1-40ff-aaf9-7b4fcab14450 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:30.988772 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:31.016244 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b7bfc7-833b-4160-a905-7ff4a7899b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:31.017349 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7bfc7-833b-4160-a905-7ff4a7899b88 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:31.017834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7bfc7-833b-4160-a905-7ff4a7899b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:31.018347 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7bfc7-833b-4160-a905-7ff4a7899b88 None admin] PATH_INFO: `/v3/projects/1509e8ef0c6042559698c40a7a07a646` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:31.020644 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b7bfc7-833b-4160-a905-7ff4a7899b88 None admin] RBAC: Authorizing `identity:get_project(project_id=1509e8ef0c6042559698c40a7a07a646)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:31.022357 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b7bfc7-833b-4160-a905-7ff4a7899b88 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:31.024749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 183/378] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:30 2022] GET /identity/v3/projects/1509e8ef0c6042559698c40a7a07a646 => generated 344 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:31.030225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea7b44b-2751-4bc9-9d02-0cf0e006f5df None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:31.141805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:31.147879 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea7b44b-2751-4bc9-9d02-0cf0e006f5df None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:31.189944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe04df-1241-4cef-be21-f26a4fb61631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:31.191434 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe04df-1241-4cef-be21-f26a4fb61631 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:31.191926 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe04df-1241-4cef-be21-f26a4fb61631 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:31.192390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe04df-1241-4cef-be21-f26a4fb61631 None admin] PATH_INFO: `/v3/groups/3a94bbd0a8d746a18b250f92578f74a8` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:31.197997 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe04df-1241-4cef-be21-f26a4fb61631 None admin] RBAC: Authorizing `identity:get_group(group_id=3a94bbd0a8d746a18b250f92578f74a8)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:31.199699 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe04df-1241-4cef-be21-f26a4fb61631 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:31.202418 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 196/379] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 31 00:07:31 2022] GET /identity/v3/groups/3a94bbd0a8d746a18b250f92578f74a8 => generated 229 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:31.208512 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b7bfc7-833b-4160-a905-7ff4a7899b88 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:31.268481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45d788-4d23-42cd-982d-5f4284867a3e None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:31.269152 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45d788-4d23-42cd-982d-5f4284867a3e None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:31.269649 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45d788-4d23-42cd-982d-5f4284867a3e None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:31.271209 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 197/380] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:07:31.277093 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2a981-b618-426d-a17b-5ed76a9e0e0d None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:31.277426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2a981-b618-426d-a17b-5ed76a9e0e0d None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:31.277647 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2a981-b618-426d-a17b-5ed76a9e0e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:31.316169 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-6da2a981-b618-426d-a17b-5ed76a9e0e0d None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:31.317797 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-6da2a981-b618-426d-a17b-5ed76a9e0e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:31.399535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b7bfc7-833b-4160-a905-7ff4a7899b88 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:31.437794 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 198/381] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:07:31.448173 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2003b24-ea5c-439a-b415-489b9cae3279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:31.448794 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2003b24-ea5c-439a-b415-489b9cae3279 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:31.449071 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2003b24-ea5c-439a-b415-489b9cae3279 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:31.449340 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2003b24-ea5c-439a-b415-489b9cae3279 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:31.450078 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2003b24-ea5c-439a-b415-489b9cae3279 None admin] RBAC: Adding query filter params (group.id=3a94bbd0a8d746a18b250f92578f74a8, role.id=34df06ce84ad4cf8b3bd1a4490228017, scope.project.id=1509e8ef0c6042559698c40a7a07a646) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:31.450939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2003b24-ea5c-439a-b415-489b9cae3279 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:31.452574 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2003b24-ea5c-439a-b415-489b9cae3279 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:31.458619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 184/382] 10.222.0.67 () {56 vars in 1376 bytes} [Wed Aug 31 00:07:31 2022] GET /identity/v3/role_assignments?group.id=3a94bbd0a8d746a18b250f92578f74a8&scope.project.id=1509e8ef0c6042559698c40a7a07a646&role.id=34df06ce84ad4cf8b3bd1a4490228017 => generated 260 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:31.753750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2a981-b618-426d-a17b-5ed76a9e0e0d None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:31.852591 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:31.860456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2a981-b618-426d-a17b-5ed76a9e0e0d None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:31.896585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03085098-d4ef-4461-b950-1e6273013d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:31.897224 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03085098-d4ef-4461-b950-1e6273013d83 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:31.897502 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03085098-d4ef-4461-b950-1e6273013d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:31.897770 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03085098-d4ef-4461-b950-1e6273013d83 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:31.908921 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03085098-d4ef-4461-b950-1e6273013d83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:31.913829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-03085098-d4ef-4461-b950-1e6273013d83 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:31.913829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:31.913829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:31.913829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:31.913829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:31.913829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:31.913829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:31.913829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:31.913829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:31.913829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:31.913829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:31.913829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:31.913829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:31.913829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:31.913829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:31.913829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:31.914799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:31.914799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:31.914799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:31.914799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:31.914799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:31.914799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:31.914799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:31.914799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:31.914799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:31.914799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:31.914799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:31.914799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:31.914799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:31.914799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:31.914799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:31.914799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:31.916009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:31.916009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:31.916009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:31.916009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:31.916009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:31.916009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:31.916009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:31.916009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:31.916009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:31.916009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:31.916009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:31.916009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:31.916009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 199/383] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:07:31.923028 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2003b24-ea5c-439a-b415-489b9cae3279 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:32.018548 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:32.090547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2003b24-ea5c-439a-b415-489b9cae3279 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:32.136049 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d65c6c-d0ea-42d5-951c-fe7d01993612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:32.137048 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d65c6c-d0ea-42d5-951c-fe7d01993612 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:32.137531 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d65c6c-d0ea-42d5-951c-fe7d01993612 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:32.138195 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d65c6c-d0ea-42d5-951c-fe7d01993612 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:32.139341 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d65c6c-d0ea-42d5-951c-fe7d01993612 None admin] RBAC: Adding query filter params (name=member) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:32.139920 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d65c6c-d0ea-42d5-951c-fe7d01993612 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:32.141515 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d65c6c-d0ea-42d5-951c-fe7d01993612 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:32.156094 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 185/384] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:07:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:32.161416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03085098-d4ef-4461-b950-1e6273013d83 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:32.287620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03085098-d4ef-4461-b950-1e6273013d83 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:32.326166 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526a430-1f76-4254-bfba-aafbc67b2ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:32.326806 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526a430-1f76-4254-bfba-aafbc67b2ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:32.327116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526a430-1f76-4254-bfba-aafbc67b2ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:32.327393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526a430-1f76-4254-bfba-aafbc67b2ca9 None admin] PATH_INFO: `/v3/groups/dd13637587354a10893278b6d62d73ca` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:32.328528 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526a430-1f76-4254-bfba-aafbc67b2ca9 None admin] RBAC: Authorizing `identity:get_group(group_id=dd13637587354a10893278b6d62d73ca)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:32.329971 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526a430-1f76-4254-bfba-aafbc67b2ca9 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:32.331868 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 200/385] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 31 00:07:32 2022] GET /identity/v3/groups/dd13637587354a10893278b6d62d73ca => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:32.337443 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d65c6c-d0ea-42d5-951c-fe7d01993612 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:32.366655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe881c-c7ed-45bf-915e-51d06fd0cd7f None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:32.366962 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe881c-c7ed-45bf-915e-51d06fd0cd7f None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:32.367241 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe881c-c7ed-45bf-915e-51d06fd0cd7f None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:32.368726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 201/386] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:32.373584 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025541c2-8545-4ca3-ae2e-241a4603ee7b None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:32.373864 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025541c2-8545-4ca3-ae2e-241a4603ee7b None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:32.374138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025541c2-8545-4ca3-ae2e-241a4603ee7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:32.407128 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-025541c2-8545-4ca3-ae2e-241a4603ee7b None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:32.408336 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-025541c2-8545-4ca3-ae2e-241a4603ee7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:32.444980 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:32.517302 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 202/387] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:07:32.538862 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d65c6c-d0ea-42d5-951c-fe7d01993612 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:32.555909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:32.558925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d1908-8040-4f53-88d0-d5206fcd9d08 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:32.559465 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d1908-8040-4f53-88d0-d5206fcd9d08 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:32.561110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d1908-8040-4f53-88d0-d5206fcd9d08 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:32.565675 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 203/388] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:32.571890 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b67737-ebc4-4bcd-94b4-c4cc86a0eacb None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:32.572405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b67737-ebc4-4bcd-94b4-c4cc86a0eacb None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:32.574000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b67737-ebc4-4bcd-94b4-c4cc86a0eacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:32.596207 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-58b67737-ebc4-4bcd-94b4-c4cc86a0eacb None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:32.598735 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-58b67737-ebc4-4bcd-94b4-c4cc86a0eacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:32.614323 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbc54a-092a-42eb-8075-606ae56c2453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:32.614985 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbc54a-092a-42eb-8075-606ae56c2453 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:32.615291 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbc54a-092a-42eb-8075-606ae56c2453 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:32.615569 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbc54a-092a-42eb-8075-606ae56c2453 None admin] PATH_INFO: `/v3/projects/9eb64431e703423581c08ff8f8378eb3` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:32.618113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbc54a-092a-42eb-8075-606ae56c2453 None admin] RBAC: Authorizing `identity:get_project(project_id=9eb64431e703423581c08ff8f8378eb3)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:32.619712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbc54a-092a-42eb-8075-606ae56c2453 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:32.621666 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 186/389] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:32 2022] GET /identity/v3/projects/9eb64431e703423581c08ff8f8378eb3 => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:32.629004 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbc54a-092a-42eb-8075-606ae56c2453 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:32.689409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 204/390] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:32.757466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbc54a-092a-42eb-8075-606ae56c2453 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:32.812754 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470c233-3243-482b-abd3-5f5ffa2770fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:32.813346 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470c233-3243-482b-abd3-5f5ffa2770fb None admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:32.813628 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470c233-3243-482b-abd3-5f5ffa2770fb None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:32.813961 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470c233-3243-482b-abd3-5f5ffa2770fb None admin] PATH_INFO: `/v3/projects/9eb64431e703423581c08ff8f8378eb3/groups/dd13637587354a10893278b6d62d73ca/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:32.819841 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470c233-3243-482b-abd3-5f5ffa2770fb None admin] RBAC: Authorizing `identity:create_grant(project_id=9eb64431e703423581c08ff8f8378eb3, group_id=dd13637587354a10893278b6d62d73ca, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:32.821289 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470c233-3243-482b-abd3-5f5ffa2770fb None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:32.844330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 187/391] 10.222.0.67 () {58 vars in 1442 bytes} [Wed Aug 31 00:07:32 2022] PUT /identity/v3/projects/9eb64431e703423581c08ff8f8378eb3/groups/dd13637587354a10893278b6d62d73ca/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:07:32.890969 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b67737-ebc4-4bcd-94b4-c4cc86a0eacb None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:33.007388 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470c233-3243-482b-abd3-5f5ffa2770fb None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:33.023998 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:33.068745 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b67737-ebc4-4bcd-94b4-c4cc86a0eacb None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:33.089901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:33.128659 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043a85f-7279-45e6-a5a3-6e4c3f4df5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:33.129654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043a85f-7279-45e6-a5a3-6e4c3f4df5e4 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:33.130078 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043a85f-7279-45e6-a5a3-6e4c3f4df5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:33.130557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043a85f-7279-45e6-a5a3-6e4c3f4df5e4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:33.142792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8043a85f-7279-45e6-a5a3-6e4c3f4df5e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:33.148407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-8043a85f-7279-45e6-a5a3-6e4c3f4df5e4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:07:33.148407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:33.148407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:33.148407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:33.148407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:33.148407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:33.148407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:33.148407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:33.148407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:33.148407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:33.148407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:33.148407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:33.148407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:33.148407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:33.148407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:33.148407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:33.149459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:33.149459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:33.149459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:33.149459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:33.149459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:33.149459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:33.149459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:33.149459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:33.149459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:33.149459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:33.149459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:33.149459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:33.149459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:33.149459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:33.149459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:33.149459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:33.150340 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:33.150340 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:33.150340 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:33.150340 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:33.150340 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:33.150340 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:33.150340 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:33.150340 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:33.150340 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:33.150340 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:33.150340 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:07:33.150340 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:33.152641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 205/392] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 31 00:07:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:33.159708 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043a85f-7279-45e6-a5a3-6e4c3f4df5e4 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:33.174723 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470c233-3243-482b-abd3-5f5ffa2770fb None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:33.218208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20ad0c-a1b0-49bd-ad2e-408188a0c932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:33.218868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20ad0c-a1b0-49bd-ad2e-408188a0c932 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:33.219183 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20ad0c-a1b0-49bd-ad2e-408188a0c932 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:33.219460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20ad0c-a1b0-49bd-ad2e-408188a0c932 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:33.231430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb20ad0c-a1b0-49bd-ad2e-408188a0c932 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:33.248128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-eb20ad0c-a1b0-49bd-ad2e-408188a0c932 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:33.248128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:33.248128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:33.248128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:33.248128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:33.248128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:33.248128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:33.248128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:33.248128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:33.248128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:33.248128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:33.248128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:33.248128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:33.248128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:33.248128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:33.248128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:33.249260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:33.249260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:33.249260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:33.249260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:33.249260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:33.249260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:33.249260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:33.249260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:33.249260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:33.249260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:33.249260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:33.249260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:33.249260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:33.249260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:33.249260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:33.249260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:33.250456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:33.250456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:33.250456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:33.250456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:33.250456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:33.250456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:33.250456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:33.250456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:33.250456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:33.250456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:33.250456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:33.250456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:33.250456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 188/393] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:07:33.256204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20ad0c-a1b0-49bd-ad2e-408188a0c932 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:33.279198 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043a85f-7279-45e6-a5a3-6e4c3f4df5e4 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:33.316754 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf968977-b8ad-42af-be0e-e4154e2fc059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:33.318056 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf968977-b8ad-42af-be0e-e4154e2fc059 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:33.318479 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf968977-b8ad-42af-be0e-e4154e2fc059 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:33.318949 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf968977-b8ad-42af-be0e-e4154e2fc059 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:33.319814 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf968977-b8ad-42af-be0e-e4154e2fc059 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:33.320370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf968977-b8ad-42af-be0e-e4154e2fc059 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:33.321887 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf968977-b8ad-42af-be0e-e4154e2fc059 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:33.331826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 206/394] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:07:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:33.337451 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf968977-b8ad-42af-be0e-e4154e2fc059 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:33.342764 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:33.443705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf968977-b8ad-42af-be0e-e4154e2fc059 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:33.451740 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:33.457938 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20ad0c-a1b0-49bd-ad2e-408188a0c932 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:33.488576 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b372d3-ba07-466a-8910-e7f5a744c8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:33.489770 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b372d3-ba07-466a-8910-e7f5a744c8a5 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:33.491121 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b372d3-ba07-466a-8910-e7f5a744c8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:33.492299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b372d3-ba07-466a-8910-e7f5a744c8a5 None admin] PATH_INFO: `/v3/groups/dd13637587354a10893278b6d62d73ca` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:33.493802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b372d3-ba07-466a-8910-e7f5a744c8a5 None admin] RBAC: Authorizing `identity:get_group(group_id=dd13637587354a10893278b6d62d73ca)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:33.496818 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b372d3-ba07-466a-8910-e7f5a744c8a5 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:33.500899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 207/395] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 31 00:07:33 2022] GET /identity/v3/groups/dd13637587354a10893278b6d62d73ca => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:33.506591 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b372d3-ba07-466a-8910-e7f5a744c8a5 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:33.529588 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89adc9cf-f551-4b05-96f6-2cca35728969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:33.530190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89adc9cf-f551-4b05-96f6-2cca35728969 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:33.530647 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89adc9cf-f551-4b05-96f6-2cca35728969 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:33.531073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89adc9cf-f551-4b05-96f6-2cca35728969 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:33.531893 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89adc9cf-f551-4b05-96f6-2cca35728969 None admin] RBAC: Adding query filter params (name=member) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:33.532334 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89adc9cf-f551-4b05-96f6-2cca35728969 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:33.533849 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89adc9cf-f551-4b05-96f6-2cca35728969 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:33.551104 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 189/396] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:07:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:33.556463 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89adc9cf-f551-4b05-96f6-2cca35728969 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:33.591851 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:33.616717 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b372d3-ba07-466a-8910-e7f5a744c8a5 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:33.692343 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97fa8a-94c2-46e3-ae4b-73d2186ec0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:33.693251 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97fa8a-94c2-46e3-ae4b-73d2186ec0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:33.693910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97fa8a-94c2-46e3-ae4b-73d2186ec0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:33.695179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97fa8a-94c2-46e3-ae4b-73d2186ec0a4 None admin] PATH_INFO: `/v3/projects/9eb64431e703423581c08ff8f8378eb3` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:33.697399 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da97fa8a-94c2-46e3-ae4b-73d2186ec0a4 None admin] RBAC: Authorizing `identity:get_project(project_id=9eb64431e703423581c08ff8f8378eb3)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:33.699206 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da97fa8a-94c2-46e3-ae4b-73d2186ec0a4 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:33.703302 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 208/397] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:33 2022] GET /identity/v3/projects/9eb64431e703423581c08ff8f8378eb3 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:33.712576 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97fa8a-94c2-46e3-ae4b-73d2186ec0a4 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:33.738749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89adc9cf-f551-4b05-96f6-2cca35728969 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:33.783028 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df8927-d844-4e22-92b8-1d29c2832c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:33.783683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df8927-d844-4e22-92b8-1d29c2832c61 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:33.783963 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df8927-d844-4e22-92b8-1d29c2832c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:33.784233 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df8927-d844-4e22-92b8-1d29c2832c61 None admin] PATH_INFO: `/v3/groups/dd13637587354a10893278b6d62d73ca` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:33.785379 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df8927-d844-4e22-92b8-1d29c2832c61 None admin] RBAC: Authorizing `identity:get_group(group_id=dd13637587354a10893278b6d62d73ca)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:33.786806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df8927-d844-4e22-92b8-1d29c2832c61 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:33.791084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 190/398] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 31 00:07:33 2022] GET /identity/v3/groups/dd13637587354a10893278b6d62d73ca => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:33.791084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74684cc9-e1b7-4686-92c5-24ffbdb198e2 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:33.791084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74684cc9-e1b7-4686-92c5-24ffbdb198e2 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:33.791084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74684cc9-e1b7-4686-92c5-24ffbdb198e2 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:33.791820 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 191/399] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:33.792822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26a9da-e77d-461b-bfec-213e8d182d7c None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:33.793097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26a9da-e77d-461b-bfec-213e8d182d7c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:33.793365 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26a9da-e77d-461b-bfec-213e8d182d7c None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:33.795701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 192/400] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:33.796272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc26a9da-e77d-461b-bfec-213e8d182d7c None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:33.824420 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:33.852744 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97fa8a-94c2-46e3-ae4b-73d2186ec0a4 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:33.890206 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed68d58-5a7c-49c8-93cf-841c99e968f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:33.890875 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed68d58-5a7c-49c8-93cf-841c99e968f8 None admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:33.891189 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed68d58-5a7c-49c8-93cf-841c99e968f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:33.891472 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed68d58-5a7c-49c8-93cf-841c99e968f8 None admin] PATH_INFO: `/v3/projects/9eb64431e703423581c08ff8f8378eb3/groups/dd13637587354a10893278b6d62d73ca/roles/4d91c00f40304b9e9ef4e6c89f46e5c5` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:33.893790 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed68d58-5a7c-49c8-93cf-841c99e968f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=9eb64431e703423581c08ff8f8378eb3, group_id=dd13637587354a10893278b6d62d73ca, role_id=4d91c00f40304b9e9ef4e6c89f46e5c5)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:33.895282 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed68d58-5a7c-49c8-93cf-841c99e968f8 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:33.907447 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:33.912169 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc26a9da-e77d-461b-bfec-213e8d182d7c None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:33.912988 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 209/401] 10.222.0.67 () {58 vars in 1442 bytes} [Wed Aug 31 00:07:33 2022] PUT /identity/v3/projects/9eb64431e703423581c08ff8f8378eb3/groups/dd13637587354a10893278b6d62d73ca/roles/4d91c00f40304b9e9ef4e6c89f46e5c5 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:07:33.917251 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea8b6b-cbfe-49af-b0d7-824ec8ea82d8 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:33.917669 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea8b6b-cbfe-49af-b0d7-824ec8ea82d8 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:33.918015 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea8b6b-cbfe-49af-b0d7-824ec8ea82d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:33.938076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-07ea8b6b-cbfe-49af-b0d7-824ec8ea82d8 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:33.939551 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-07ea8b6b-cbfe-49af-b0d7-824ec8ea82d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:33.962860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794bda43-0dcb-40da-98a9-f476246d8685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:33.963905 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bda43-0dcb-40da-98a9-f476246d8685 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:33.964365 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bda43-0dcb-40da-98a9-f476246d8685 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:33.964890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bda43-0dcb-40da-98a9-f476246d8685 None admin] PATH_INFO: `/v3/projects/a8b13074ade34e4d944896277be959f5` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:33.967324 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794bda43-0dcb-40da-98a9-f476246d8685 None admin] RBAC: Authorizing `identity:get_project(project_id=a8b13074ade34e4d944896277be959f5)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:33.975153 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794bda43-0dcb-40da-98a9-f476246d8685 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:33.983060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 193/402] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:33 2022] GET /identity/v3/projects/a8b13074ade34e4d944896277be959f5 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:33.984923 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f3f69-f162-4529-b10f-f89d1d451b71 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:33.985531 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f3f69-f162-4529-b10f-f89d1d451b71 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:33.986009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f3f69-f162-4529-b10f-f89d1d451b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:34.014513 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-514f3f69-f162-4529-b10f-f89d1d451b71 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:34.016432 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-514f3f69-f162-4529-b10f-f89d1d451b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:34.031071 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 210/403] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:34.031071 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea8b6b-cbfe-49af-b0d7-824ec8ea82d8 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:34.147163 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 194/404] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:34.165898 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea8b6b-cbfe-49af-b0d7-824ec8ea82d8 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:34.200075 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:34.206817 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d7558-8e44-457e-92c6-d51102af38a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:34.207952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d7558-8e44-457e-92c6-d51102af38a6 None admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:34.208467 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d7558-8e44-457e-92c6-d51102af38a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:34.208977 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d7558-8e44-457e-92c6-d51102af38a6 None admin] PATH_INFO: `/v3/projects/a8b13074ade34e4d944896277be959f5/groups/dd13637587354a10893278b6d62d73ca/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:34.211251 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32d7558-8e44-457e-92c6-d51102af38a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8b13074ade34e4d944896277be959f5, group_id=dd13637587354a10893278b6d62d73ca, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:34.212897 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32d7558-8e44-457e-92c6-d51102af38a6 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:34.243874 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 211/405] 10.222.0.67 () {58 vars in 1442 bytes} [Wed Aug 31 00:07:34 2022] PUT /identity/v3/projects/a8b13074ade34e4d944896277be959f5/groups/dd13637587354a10893278b6d62d73ca/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:07:34.345850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f3f69-f162-4529-b10f-f89d1d451b71 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:34.375814 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:34.446889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d7558-8e44-457e-92c6-d51102af38a6 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:34.458080 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f3f69-f162-4529-b10f-f89d1d451b71 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:34.496216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21148c40-52e2-4ec7-9115-d437d9bfb30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:34.496875 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21148c40-52e2-4ec7-9115-d437d9bfb30d None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:34.497697 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21148c40-52e2-4ec7-9115-d437d9bfb30d None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:34.498180 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21148c40-52e2-4ec7-9115-d437d9bfb30d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:34.510251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21148c40-52e2-4ec7-9115-d437d9bfb30d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:34.515770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-21148c40-52e2-4ec7-9115-d437d9bfb30d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:07:34.515770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:34.515770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:34.515770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:34.515770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:34.515770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:34.515770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:34.515770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:34.515770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:34.515770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:34.515770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:34.515770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:34.515770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:34.515770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:34.515770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:34.515770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:34.516877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:34.516877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:34.516877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:34.516877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:34.516877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:34.516877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:34.516877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:34.516877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:34.516877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:34.516877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:34.516877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:34.516877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:34.516877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:34.516877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:34.516877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:34.516877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:34.517870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:34.517870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:34.517870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:34.517870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:34.517870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:34.517870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:34.517870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:34.517870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:34.517870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:34.517870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:34.517870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:07:34.517870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:34.517870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 195/406] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:07:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:34.529549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21148c40-52e2-4ec7-9115-d437d9bfb30d None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:34.581618 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d7558-8e44-457e-92c6-d51102af38a6 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:34.632871 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab1106-1520-49b1-b746-b8e6f1b107f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:34.633522 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab1106-1520-49b1-b746-b8e6f1b107f5 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:34.633808 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab1106-1520-49b1-b746-b8e6f1b107f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:34.634087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab1106-1520-49b1-b746-b8e6f1b107f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:34.658097 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab1106-1520-49b1-b746-b8e6f1b107f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:34.663842 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:34.673538 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-57ab1106-1520-49b1-b746-b8e6f1b107f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:07:34.673538 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:34.673538 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:34.673538 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:34.673538 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:34.673538 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:34.673538 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:34.673538 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:34.673538 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:34.673538 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:34.673538 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:34.673538 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:34.673538 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:34.673538 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:34.673538 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:34.673538 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:34.676055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:34.676055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:34.676055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:34.676055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:34.676055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:34.676055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:34.676055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:34.676055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:34.676055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:34.676055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:34.676055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:34.676055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:34.676055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:34.676055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:34.676055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:34.676055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:34.678000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:34.678000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:34.678000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:34.678000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:34.678000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:34.678000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:34.678000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:34.678000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:34.678000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:34.678000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:34.678000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:07:34.678000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:34.678000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 212/407] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 31 00:07:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:34.687062 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab1106-1520-49b1-b746-b8e6f1b107f5 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:34.694772 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21148c40-52e2-4ec7-9115-d437d9bfb30d None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:34.765726 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d39be6-65d8-4d2c-9c2e-f0e5e1748511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:34.766257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d39be6-65d8-4d2c-9c2e-f0e5e1748511 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:34.766537 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d39be6-65d8-4d2c-9c2e-f0e5e1748511 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:34.766811 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d39be6-65d8-4d2c-9c2e-f0e5e1748511 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:34.767519 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d39be6-65d8-4d2c-9c2e-f0e5e1748511 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:34.767824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d39be6-65d8-4d2c-9c2e-f0e5e1748511 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:34.769204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d39be6-65d8-4d2c-9c2e-f0e5e1748511 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:34.779628 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:34.787807 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 196/408] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:34.792580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d39be6-65d8-4d2c-9c2e-f0e5e1748511 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:34.916403 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab1106-1520-49b1-b746-b8e6f1b107f5 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:34.932482 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d39be6-65d8-4d2c-9c2e-f0e5e1748511 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:34.974218 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5abc3-83c1-4e84-982f-2a2a07fed4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:34.974218 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5abc3-83c1-4e84-982f-2a2a07fed4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:34.974218 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5abc3-83c1-4e84-982f-2a2a07fed4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:34.974218 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5abc3-83c1-4e84-982f-2a2a07fed4e6 None admin] PATH_INFO: `/v3/groups/3a94bbd0a8d746a18b250f92578f74a8` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:34.974609 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e5abc3-83c1-4e84-982f-2a2a07fed4e6 None admin] RBAC: Authorizing `identity:get_group(group_id=3a94bbd0a8d746a18b250f92578f74a8)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:34.975848 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e5abc3-83c1-4e84-982f-2a2a07fed4e6 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:34.977805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 197/409] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 31 00:07:34 2022] GET /identity/v3/groups/3a94bbd0a8d746a18b250f92578f74a8 => generated 229 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:34.983544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5abc3-83c1-4e84-982f-2a2a07fed4e6 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:35.013462 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33993475-76ac-4c0b-9310-84e9fb052a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:35.014440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33993475-76ac-4c0b-9310-84e9fb052a13 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:35.014920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33993475-76ac-4c0b-9310-84e9fb052a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:35.015430 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33993475-76ac-4c0b-9310-84e9fb052a13 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:35.016379 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33993475-76ac-4c0b-9310-84e9fb052a13 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:35.016884 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33993475-76ac-4c0b-9310-84e9fb052a13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:35.018465 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33993475-76ac-4c0b-9310-84e9fb052a13 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:35.026946 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:35.032947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 213/410] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:07:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:35.037389 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33993475-76ac-4c0b-9310-84e9fb052a13 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:35.174496 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33993475-76ac-4c0b-9310-84e9fb052a13 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:35.191662 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:35.205916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5abc3-83c1-4e84-982f-2a2a07fed4e6 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:35.215682 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5310907e-e80d-4e49-978a-9409cbde7561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:35.217692 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310907e-e80d-4e49-978a-9409cbde7561 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:35.218192 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310907e-e80d-4e49-978a-9409cbde7561 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:35.218663 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310907e-e80d-4e49-978a-9409cbde7561 None admin] PATH_INFO: `/v3/groups/dd13637587354a10893278b6d62d73ca` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:35.220077 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5310907e-e80d-4e49-978a-9409cbde7561 None admin] RBAC: Authorizing `identity:get_group(group_id=dd13637587354a10893278b6d62d73ca)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:35.222238 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5310907e-e80d-4e49-978a-9409cbde7561 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:35.224848 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 214/411] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 31 00:07:35 2022] GET /identity/v3/groups/dd13637587354a10893278b6d62d73ca => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:35.233421 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5310907e-e80d-4e49-978a-9409cbde7561 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:35.276033 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39755f-c74c-4298-81b4-7e76ead9901f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:35.276727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39755f-c74c-4298-81b4-7e76ead9901f None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:35.277013 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39755f-c74c-4298-81b4-7e76ead9901f None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:35.277289 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39755f-c74c-4298-81b4-7e76ead9901f None admin] PATH_INFO: `/v3/projects/1509e8ef0c6042559698c40a7a07a646` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:35.279515 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da39755f-c74c-4298-81b4-7e76ead9901f None admin] RBAC: Authorizing `identity:get_project(project_id=1509e8ef0c6042559698c40a7a07a646)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:35.281164 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da39755f-c74c-4298-81b4-7e76ead9901f None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:35.282825 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 198/412] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:34 2022] GET /identity/v3/projects/1509e8ef0c6042559698c40a7a07a646 => generated 344 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:35.289295 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39755f-c74c-4298-81b4-7e76ead9901f None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:35.430428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5310907e-e80d-4e49-978a-9409cbde7561 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:35.444595 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:35.462072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39755f-c74c-4298-81b4-7e76ead9901f None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:35.475363 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61852381-70e8-4904-babc-9f51cfb5fd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:35.476253 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61852381-70e8-4904-babc-9f51cfb5fd55 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:35.476998 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61852381-70e8-4904-babc-9f51cfb5fd55 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:35.477357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61852381-70e8-4904-babc-9f51cfb5fd55 None admin] PATH_INFO: `/v3/projects/a8b13074ade34e4d944896277be959f5` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:35.478769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61852381-70e8-4904-babc-9f51cfb5fd55 None admin] RBAC: Authorizing `identity:get_project(project_id=a8b13074ade34e4d944896277be959f5)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:35.480925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61852381-70e8-4904-babc-9f51cfb5fd55 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:35.483931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 215/413] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:35 2022] GET /identity/v3/projects/a8b13074ade34e4d944896277be959f5 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:35.485187 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4051a-ec98-4664-9e66-99b6c63878b0 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:35.485479 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4051a-ec98-4664-9e66-99b6c63878b0 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:35.485755 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4051a-ec98-4664-9e66-99b6c63878b0 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:35.491299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 216/414] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:35.492165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4051a-ec98-4664-9e66-99b6c63878b0 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:35.505814 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e886225-48bf-45e5-bb11-b7614f089b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:35.506486 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e886225-48bf-45e5-bb11-b7614f089b4c None admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:35.506765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e886225-48bf-45e5-bb11-b7614f089b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:35.507065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e886225-48bf-45e5-bb11-b7614f089b4c None admin] PATH_INFO: `/v3/projects/1509e8ef0c6042559698c40a7a07a646/groups/3a94bbd0a8d746a18b250f92578f74a8/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:35.509151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e886225-48bf-45e5-bb11-b7614f089b4c None admin] RBAC: Authorizing `identity:create_grant(project_id=1509e8ef0c6042559698c40a7a07a646, group_id=3a94bbd0a8d746a18b250f92578f74a8, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:35.510582 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e886225-48bf-45e5-bb11-b7614f089b4c None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:35.526132 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 199/415] 10.222.0.67 () {58 vars in 1442 bytes} [Wed Aug 31 00:07:35 2022] PUT /identity/v3/projects/1509e8ef0c6042559698c40a7a07a646/groups/3a94bbd0a8d746a18b250f92578f74a8/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:07:35.527399 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ccef8-f7c8-45d1-b608-97a4eb518700 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:35.527687 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ccef8-f7c8-45d1-b608-97a4eb518700 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:35.527962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ccef8-f7c8-45d1-b608-97a4eb518700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:35.548857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-3e0ccef8-f7c8-45d1-b608-97a4eb518700 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:35.550121 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-3e0ccef8-f7c8-45d1-b608-97a4eb518700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:35.576268 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:35.646827 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4051a-ec98-4664-9e66-99b6c63878b0 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:35.657791 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 200/416] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:35.703556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd8d97-536d-4169-93a8-4397addc8603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:35.704249 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd8d97-536d-4169-93a8-4397addc8603 None admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:35.704531 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd8d97-536d-4169-93a8-4397addc8603 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:35.704805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd8d97-536d-4169-93a8-4397addc8603 None admin] PATH_INFO: `/v3/projects/a8b13074ade34e4d944896277be959f5/groups/dd13637587354a10893278b6d62d73ca/roles/4d91c00f40304b9e9ef4e6c89f46e5c5` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:35.713195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd8d97-536d-4169-93a8-4397addc8603 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8b13074ade34e4d944896277be959f5, group_id=dd13637587354a10893278b6d62d73ca, role_id=4d91c00f40304b9e9ef4e6c89f46e5c5)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:35.714696 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd8d97-536d-4169-93a8-4397addc8603 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:35.742474 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 217/417] 10.222.0.67 () {58 vars in 1442 bytes} [Wed Aug 31 00:07:35 2022] PUT /identity/v3/projects/a8b13074ade34e4d944896277be959f5/groups/dd13637587354a10893278b6d62d73ca/roles/4d91c00f40304b9e9ef4e6c89f46e5c5 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:07:36.011153 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ccef8-f7c8-45d1-b608-97a4eb518700 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:36.112780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:36.155219 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ccef8-f7c8-45d1-b608-97a4eb518700 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:36.202149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f3a26-a143-4a7c-8886-6c07281d1acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:36.202788 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f3a26-a143-4a7c-8886-6c07281d1acc None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:36.203080 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f3a26-a143-4a7c-8886-6c07281d1acc None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:36.203357 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f3a26-a143-4a7c-8886-6c07281d1acc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:36.219726 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f3a26-a143-4a7c-8886-6c07281d1acc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:36.224752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-019f3a26-a143-4a7c-8886-6c07281d1acc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:36.224752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:36.224752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:36.224752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:36.224752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:36.224752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:36.224752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:36.224752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:36.224752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:36.224752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:36.224752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:36.224752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:36.224752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:36.224752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:36.224752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:36.224752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:36.225635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:36.225635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:36.225635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:36.225635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:36.225635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:36.225635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:36.225635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:36.225635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:36.225635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:36.225635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:36.225635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:36.225635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:36.225635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:36.225635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:36.225635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:36.225635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:36.226430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:36.226430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:36.226430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:36.226430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:36.226430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:36.226430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:36.226430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:36.226430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:36.226430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:36.226430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:36.226430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:36.226430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:36.226430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 201/418] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:36.236169 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd8d97-536d-4169-93a8-4397addc8603 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:36.259730 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc986c3c-389a-44b8-a9dd-1f33ea90c183 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:36.260010 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc986c3c-389a-44b8-a9dd-1f33ea90c183 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:36.260192 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc986c3c-389a-44b8-a9dd-1f33ea90c183 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:36.261779 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 202/419] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:36.268815 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19acb145-aa2d-4797-8c31-5533f10e685c None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:36.269097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19acb145-aa2d-4797-8c31-5533f10e685c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:36.269374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19acb145-aa2d-4797-8c31-5533f10e685c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:36.304303 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-19acb145-aa2d-4797-8c31-5533f10e685c None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:36.305593 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-19acb145-aa2d-4797-8c31-5533f10e685c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:36.358518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd8d97-536d-4169-93a8-4397addc8603 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:36.369558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:36.396667 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d60376-98eb-4c13-8753-a54c8f4ed448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:36.398047 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d60376-98eb-4c13-8753-a54c8f4ed448 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:36.398559 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d60376-98eb-4c13-8753-a54c8f4ed448 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:36.399073 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d60376-98eb-4c13-8753-a54c8f4ed448 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:36.399953 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d60376-98eb-4c13-8753-a54c8f4ed448 None admin] RBAC: Adding query filter params (name=member) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:36.400500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d60376-98eb-4c13-8753-a54c8f4ed448 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:36.402068 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d60376-98eb-4c13-8753-a54c8f4ed448 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:36.413190 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 218/420] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:07:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:36.420652 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d60376-98eb-4c13-8753-a54c8f4ed448 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:36.454198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 203/421] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:07:36.581399 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d60376-98eb-4c13-8753-a54c8f4ed448 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:36.630437 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d8472-ff63-40fa-8c6c-a71305b6c43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:36.631064 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d8472-ff63-40fa-8c6c-a71305b6c43d None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:36.631364 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d8472-ff63-40fa-8c6c-a71305b6c43d None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:36.631639 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d8472-ff63-40fa-8c6c-a71305b6c43d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:36.681287 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:36.684337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ee3ee-2e95-4c5b-b74c-e4e55cec32b1 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:36.685315 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ee3ee-2e95-4c5b-b74c-e4e55cec32b1 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:36.685618 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ee3ee-2e95-4c5b-b74c-e4e55cec32b1 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:36.687729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 204/422] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:36.702037 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780a080-6a71-445c-950a-8ba1cfef9b25 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:36.702324 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780a080-6a71-445c-950a-8ba1cfef9b25 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:36.702578 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780a080-6a71-445c-950a-8ba1cfef9b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:36.730670 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d8472-ff63-40fa-8c6c-a71305b6c43d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:36.732240 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d8472-ff63-40fa-8c6c-a71305b6c43d None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:36.738766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-2780a080-6a71-445c-950a-8ba1cfef9b25 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:36.739970 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-2780a080-6a71-445c-950a-8ba1cfef9b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:36.840516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:36.853373 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 205/423] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:07:36.860315 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780a080-6a71-445c-950a-8ba1cfef9b25 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:36.925978 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 219/424] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:36.931308 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d8472-ff63-40fa-8c6c-a71305b6c43d None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:36.987946 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780a080-6a71-445c-950a-8ba1cfef9b25 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:37.035169 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd35f5-92e3-4a7b-951e-bdd06ecf1551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:37.036613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd35f5-92e3-4a7b-951e-bdd06ecf1551 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:37.037091 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd35f5-92e3-4a7b-951e-bdd06ecf1551 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:37.037569 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd35f5-92e3-4a7b-951e-bdd06ecf1551 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:37.050985 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd35f5-92e3-4a7b-951e-bdd06ecf1551 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:37.056778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-c5dd35f5-92e3-4a7b-951e-bdd06ecf1551 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:07:37.056778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:37.056778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:37.056778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:37.056778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:37.056778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:37.056778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:37.056778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:37.056778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:37.056778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:37.056778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:37.056778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:37.056778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:37.056778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:37.056778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:37.056778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:37.057638 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:37.057638 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:37.057638 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:37.057638 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:37.057638 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:37.057638 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:37.057638 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:37.057638 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:37.057638 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:37.057638 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:37.057638 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:37.057638 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:37.057638 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:37.057638 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:37.057638 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:37.057638 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:37.058446 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:37.058446 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:37.058446 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:37.058446 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:37.058446 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:37.058446 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:37.058446 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:37.058446 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:37.058446 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:37.058446 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:37.058446 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:07:37.058446 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:37.062040 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 206/425] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 31 00:07:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:37.068448 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd35f5-92e3-4a7b-951e-bdd06ecf1551 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:37.132327 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:37.137072 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:37.201639 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd35f5-92e3-4a7b-951e-bdd06ecf1551 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:37.237243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d8472-ff63-40fa-8c6c-a71305b6c43d None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:37.240812 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d37ec-565a-4c85-bd32-a2f5e0c216bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:37.241443 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d37ec-565a-4c85-bd32-a2f5e0c216bd None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:37.241718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d37ec-565a-4c85-bd32-a2f5e0c216bd None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:37.241988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d37ec-565a-4c85-bd32-a2f5e0c216bd None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:37.242675 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d37ec-565a-4c85-bd32-a2f5e0c216bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:37.242973 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d37ec-565a-4c85-bd32-a2f5e0c216bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:37.245615 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d37ec-565a-4c85-bd32-a2f5e0c216bd None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:37.257149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 207/426] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:07:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:37.264960 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d37ec-565a-4c85-bd32-a2f5e0c216bd None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:37.290844 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f341bd3-b0e6-48b7-91a1-8ab1305625d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:37.291753 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f341bd3-b0e6-48b7-91a1-8ab1305625d1 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:37.292041 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f341bd3-b0e6-48b7-91a1-8ab1305625d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:37.292377 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f341bd3-b0e6-48b7-91a1-8ab1305625d1 None admin] PATH_INFO: `/v3/projects/9eb64431e703423581c08ff8f8378eb3` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:37.293715 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f341bd3-b0e6-48b7-91a1-8ab1305625d1 None admin] RBAC: Authorizing `identity:get_project(project_id=9eb64431e703423581c08ff8f8378eb3)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:37.296473 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f341bd3-b0e6-48b7-91a1-8ab1305625d1 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:37.298849 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 220/427] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:36 2022] GET /identity/v3/projects/9eb64431e703423581c08ff8f8378eb3 => generated 298 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:37.299869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f341bd3-b0e6-48b7-91a1-8ab1305625d1 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:37.397689 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d37ec-565a-4c85-bd32-a2f5e0c216bd None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:37.413132 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f341bd3-b0e6-48b7-91a1-8ab1305625d1 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:37.413907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:37.438030 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd29d5d-ea41-49e1-af31-e3a1c7731a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:37.438744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd29d5d-ea41-49e1-af31-e3a1c7731a0e None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:37.439226 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd29d5d-ea41-49e1-af31-e3a1c7731a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:37.439551 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd29d5d-ea41-49e1-af31-e3a1c7731a0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:37.460619 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a39b4d-a753-4e06-b289-9c605e43fd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:37.461468 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a39b4d-a753-4e06-b289-9c605e43fd19 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:37.461902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a39b4d-a753-4e06-b289-9c605e43fd19 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:37.462322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a39b4d-a753-4e06-b289-9c605e43fd19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:37.472103 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:37.477275 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a39b4d-a753-4e06-b289-9c605e43fd19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:37.484279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-13a39b4d-a753-4e06-b289-9c605e43fd19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:37.484279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:37.484279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:37.484279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:37.484279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:37.484279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:37.484279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:37.484279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:37.484279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:37.484279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:37.484279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:37.484279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:37.484279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:37.484279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:37.484279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:37.484279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:37.485345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:37.485345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:37.485345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:37.485345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:37.485345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:37.485345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:37.485345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:37.485345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:37.485345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:37.485345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:37.485345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:37.485345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:37.485345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:37.485345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:37.485345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:37.485345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:37.486303 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:37.486303 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:37.486303 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:37.486303 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:37.486303 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:37.486303 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:37.486303 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:37.486303 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:37.486303 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:37.486303 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:37.486303 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 00:07:37.486303 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:37.493469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 221/428] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:37.493469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a39b4d-a753-4e06-b289-9c605e43fd19 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:37.532691 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd29d5d-ea41-49e1-af31-e3a1c7731a0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:37.534534 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd29d5d-ea41-49e1-af31-e3a1c7731a0e None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:37.601216 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a39b4d-a753-4e06-b289-9c605e43fd19 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:37.637345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0a6ff-d8f8-4792-a27c-693de2861ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:37.638654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0a6ff-d8f8-4792-a27c-693de2861ade None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:37.638936 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0a6ff-d8f8-4792-a27c-693de2861ade None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:37.639233 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0a6ff-d8f8-4792-a27c-693de2861ade None admin] PATH_INFO: `/v3/groups/dd13637587354a10893278b6d62d73ca` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:37.640440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0a6ff-d8f8-4792-a27c-693de2861ade None admin] RBAC: Authorizing `identity:get_group(group_id=dd13637587354a10893278b6d62d73ca)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:37.641893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0a6ff-d8f8-4792-a27c-693de2861ade None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:37.643954 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 222/429] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 31 00:07:37 2022] GET /identity/v3/groups/dd13637587354a10893278b6d62d73ca => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:37.644828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0a6ff-d8f8-4792-a27c-693de2861ade None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:37.664632 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 208/430] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:37.665766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd29d5d-ea41-49e1-af31-e3a1c7731a0e None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:37.685544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:37.717044 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:37.758645 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0a6ff-d8f8-4792-a27c-693de2861ade None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:37.789237 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd29d5d-ea41-49e1-af31-e3a1c7731a0e None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:37.795954 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f651e-fe83-4db2-b826-5ad8a9c33ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:37.796577 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f651e-fe83-4db2-b826-5ad8a9c33ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:37.796854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f651e-fe83-4db2-b826-5ad8a9c33ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:37.797133 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f651e-fe83-4db2-b826-5ad8a9c33ab6 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:37.797807 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f651e-fe83-4db2-b826-5ad8a9c33ab6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:37.798109 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f651e-fe83-4db2-b826-5ad8a9c33ab6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:37.799545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f651e-fe83-4db2-b826-5ad8a9c33ab6 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:37.807878 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 223/431] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:07:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:37.808682 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f651e-fe83-4db2-b826-5ad8a9c33ab6 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:37.831183 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55296db0-f158-4012-95b7-f85e9f1b95ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:37.831786 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55296db0-f158-4012-95b7-f85e9f1b95ce None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:37.832064 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55296db0-f158-4012-95b7-f85e9f1b95ce None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:37.832330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55296db0-f158-4012-95b7-f85e9f1b95ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:37.833048 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55296db0-f158-4012-95b7-f85e9f1b95ce None admin] RBAC: Adding query filter params (group.id=dd13637587354a10893278b6d62d73ca, role.id=cc8bd97c7b9745d1a52746650a2e49b9, scope.project.id=9eb64431e703423581c08ff8f8378eb3) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:37.833354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55296db0-f158-4012-95b7-f85e9f1b95ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:37.834764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55296db0-f158-4012-95b7-f85e9f1b95ce None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:37.840467 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 209/432] 10.222.0.67 () {56 vars in 1376 bytes} [Wed Aug 31 00:07:37 2022] GET /identity/v3/role_assignments?group.id=dd13637587354a10893278b6d62d73ca&scope.project.id=9eb64431e703423581c08ff8f8378eb3&role.id=cc8bd97c7b9745d1a52746650a2e49b9 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:37.842049 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f1a68-5374-4174-891e-602ba048e369 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:37.842332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f1a68-5374-4174-891e-602ba048e369 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:37.842607 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f1a68-5374-4174-891e-602ba048e369 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:37.847867 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 210/433] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:37.847867 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f1a68-5374-4174-891e-602ba048e369 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:37.932985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f651e-fe83-4db2-b826-5ad8a9c33ab6 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:37.951233 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:37.975655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:37.984141 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636313ed-d8c9-4250-be89-57194bb091e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:37.985179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636313ed-d8c9-4250-be89-57194bb091e1 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:37.985538 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636313ed-d8c9-4250-be89-57194bb091e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:37.985882 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636313ed-d8c9-4250-be89-57194bb091e1 None admin] PATH_INFO: `/v3/projects/9eb64431e703423581c08ff8f8378eb3` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:37.988398 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636313ed-d8c9-4250-be89-57194bb091e1 None admin] RBAC: Authorizing `identity:get_project(project_id=9eb64431e703423581c08ff8f8378eb3)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:37.990301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636313ed-d8c9-4250-be89-57194bb091e1 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:37.990816 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f1a68-5374-4174-891e-602ba048e369 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:37.995405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 224/434] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:37 2022] GET /identity/v3/projects/9eb64431e703423581c08ff8f8378eb3 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:37.996871 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90059c63-ab31-49b7-a8d7-83f14c8aeecc None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:37.998817 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90059c63-ab31-49b7-a8d7-83f14c8aeecc None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:37.999359 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90059c63-ab31-49b7-a8d7-83f14c8aeecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:38.022952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-90059c63-ab31-49b7-a8d7-83f14c8aeecc None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:38.024538 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-90059c63-ab31-49b7-a8d7-83f14c8aeecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:38.044043 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e363c7-36d1-480c-8bd2-398871386969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:38.045305 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e363c7-36d1-480c-8bd2-398871386969 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:38.045588 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e363c7-36d1-480c-8bd2-398871386969 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:38.045870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e363c7-36d1-480c-8bd2-398871386969 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:38.114409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 225/435] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:38.115704 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90059c63-ab31-49b7-a8d7-83f14c8aeecc None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:38.143543 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e363c7-36d1-480c-8bd2-398871386969 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:38.145375 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e363c7-36d1-480c-8bd2-398871386969 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:38.238049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90059c63-ab31-49b7-a8d7-83f14c8aeecc None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:38.261004 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:38.268700 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:38.282272 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9c6f7-f397-499a-a08f-e261c5545689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:38.283254 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9c6f7-f397-499a-a08f-e261c5545689 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:38.283742 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9c6f7-f397-499a-a08f-e261c5545689 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:38.284198 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9c6f7-f397-499a-a08f-e261c5545689 None admin] PATH_INFO: `/v3/groups/dd13637587354a10893278b6d62d73ca` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:38.285504 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe9c6f7-f397-499a-a08f-e261c5545689 None admin] RBAC: Authorizing `identity:get_group(group_id=dd13637587354a10893278b6d62d73ca)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:38.286942 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe9c6f7-f397-499a-a08f-e261c5545689 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:38.288561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 226/436] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 31 00:07:38 2022] GET /identity/v3/groups/dd13637587354a10893278b6d62d73ca => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:38.289710 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc399ca-c5d0-464c-a08e-e4a1cfd015f6 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:38.290062 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc399ca-c5d0-464c-a08e-e4a1cfd015f6 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:38.290339 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc399ca-c5d0-464c-a08e-e4a1cfd015f6 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:38.292169 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 227/437] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:38.295635 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc399ca-c5d0-464c-a08e-e4a1cfd015f6 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:38.301537 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 211/438] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:38.303123 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738bfd7-a6f0-405a-903c-d89fbcb31208 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:38.303590 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738bfd7-a6f0-405a-903c-d89fbcb31208 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:38.304126 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738bfd7-a6f0-405a-903c-d89fbcb31208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:38.327012 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-9738bfd7-a6f0-405a-903c-d89fbcb31208 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:38.329188 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-9738bfd7-a6f0-405a-903c-d89fbcb31208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:38.412746 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 212/439] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:38.413587 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738bfd7-a6f0-405a-903c-d89fbcb31208 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:38.430922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc399ca-c5d0-464c-a08e-e4a1cfd015f6 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:38.475021 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718ef4e-fa31-4948-a2f0-c5a63cad5efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:38.475644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718ef4e-fa31-4948-a2f0-c5a63cad5efc None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:38.475936 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718ef4e-fa31-4948-a2f0-c5a63cad5efc None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:38.476181 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718ef4e-fa31-4948-a2f0-c5a63cad5efc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:38.476899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7718ef4e-fa31-4948-a2f0-c5a63cad5efc None admin] RBAC: Adding query filter params (group.id=dd13637587354a10893278b6d62d73ca, role.id=4d91c00f40304b9e9ef4e6c89f46e5c5, scope.project.id=9eb64431e703423581c08ff8f8378eb3) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:38.477200 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7718ef4e-fa31-4948-a2f0-c5a63cad5efc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:38.478617 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7718ef4e-fa31-4948-a2f0-c5a63cad5efc None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:38.485351 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 228/440] 10.222.0.67 () {56 vars in 1376 bytes} [Wed Aug 31 00:07:38 2022] GET /identity/v3/role_assignments?group.id=dd13637587354a10893278b6d62d73ca&scope.project.id=9eb64431e703423581c08ff8f8378eb3&role.id=4d91c00f40304b9e9ef4e6c89f46e5c5 => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:38.486378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718ef4e-fa31-4948-a2f0-c5a63cad5efc None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:38.532663 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738bfd7-a6f0-405a-903c-d89fbcb31208 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:38.546880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:38.567388 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:38.585668 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc72063-c7b7-4ed3-b373-c4898086d114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:38.586652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc72063-c7b7-4ed3-b373-c4898086d114 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:38.586947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc72063-c7b7-4ed3-b373-c4898086d114 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:38.587412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc72063-c7b7-4ed3-b373-c4898086d114 None admin] PATH_INFO: `/v3/projects/a8b13074ade34e4d944896277be959f5` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:38.589310 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc72063-c7b7-4ed3-b373-c4898086d114 None admin] RBAC: Authorizing `identity:get_project(project_id=a8b13074ade34e4d944896277be959f5)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:38.591797 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc72063-c7b7-4ed3-b373-c4898086d114 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:38.594408 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 213/441] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:38 2022] GET /identity/v3/projects/a8b13074ade34e4d944896277be959f5 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:38.599916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc72063-c7b7-4ed3-b373-c4898086d114 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:38.622641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718ef4e-fa31-4948-a2f0-c5a63cad5efc None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:38.679648 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e6e7c-e0ab-4923-ba7f-f921abf84708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:38.680669 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e6e7c-e0ab-4923-ba7f-f921abf84708 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:38.681052 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e6e7c-e0ab-4923-ba7f-f921abf84708 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:38.681336 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e6e7c-e0ab-4923-ba7f-f921abf84708 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:38.693708 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e6e7c-e0ab-4923-ba7f-f921abf84708 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:38.699427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-eb4e6e7c-e0ab-4923-ba7f-f921abf84708 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:07:38.699427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:38.699427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:38.699427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:38.699427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:38.699427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:38.699427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:38.699427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:38.699427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:38.699427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:38.699427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:38.699427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:38.699427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:38.699427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:38.699427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:38.699427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:38.700874 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:38.700874 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:38.700874 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:38.700874 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:38.700874 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:38.700874 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:38.700874 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:38.700874 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:38.700874 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:38.700874 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:38.700874 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:38.700874 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:38.700874 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:38.700874 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:38.700874 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:38.700874 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:38.702074 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:38.702074 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:38.702074 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:38.702074 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:38.702074 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:38.702074 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:38.702074 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:38.702074 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:38.702074 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:38.702074 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:38.702074 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:07:38.702074 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:38.702074 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 229/442] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:07:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:38.702074 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e6e7c-e0ab-4923-ba7f-f921abf84708 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:38.735499 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc72063-c7b7-4ed3-b373-c4898086d114 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:38.779014 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe90901-d699-4537-8a9f-35d0c7c4a733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:38.779935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe90901-d699-4537-8a9f-35d0c7c4a733 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:38.780220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe90901-d699-4537-8a9f-35d0c7c4a733 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:38.780493 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe90901-d699-4537-8a9f-35d0c7c4a733 None admin] PATH_INFO: `/v3/groups/dd13637587354a10893278b6d62d73ca` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:38.782411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe90901-d699-4537-8a9f-35d0c7c4a733 None admin] RBAC: Authorizing `identity:get_group(group_id=dd13637587354a10893278b6d62d73ca)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:38.783823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe90901-d699-4537-8a9f-35d0c7c4a733 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:38.785388 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 214/443] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 31 00:07:38 2022] GET /identity/v3/groups/dd13637587354a10893278b6d62d73ca => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:38.787915 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe90901-d699-4537-8a9f-35d0c7c4a733 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:38.824319 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e6e7c-e0ab-4923-ba7f-f921abf84708 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:38.843373 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:38.846683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:38.863009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f14922-184f-4d52-87e7-200595d15dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:38.863668 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f14922-184f-4d52-87e7-200595d15dce None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:38.863946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f14922-184f-4d52-87e7-200595d15dce None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:38.864212 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f14922-184f-4d52-87e7-200595d15dce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:38.877987 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f14922-184f-4d52-87e7-200595d15dce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:38.883541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-08f14922-184f-4d52-87e7-200595d15dce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:07:38.883541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:38.883541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:38.883541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:38.883541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:38.883541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:38.883541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:38.883541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:38.883541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:38.883541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:38.883541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:38.883541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:38.883541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:38.883541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:38.883541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:38.883541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:38.884508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:38.884508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:38.884508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:38.884508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:38.884508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:38.884508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:38.884508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:38.884508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:38.884508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:38.884508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:38.884508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:38.884508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:38.884508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:38.884508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:38.884508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:38.884508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:38.891456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:38.891456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:38.891456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:38.891456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:38.891456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:38.891456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:38.891456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:38.891456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:38.891456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:38.891456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:38.891456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:07:38.891456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:38.891456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 230/444] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 31 00:07:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:38.891456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f14922-184f-4d52-87e7-200595d15dce None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:38.912945 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe90901-d699-4537-8a9f-35d0c7c4a733 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:38.947009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b6dfd-4f74-4fba-a364-aa77aed72e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:38.947843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b6dfd-4f74-4fba-a364-aa77aed72e57 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:38.948126 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b6dfd-4f74-4fba-a364-aa77aed72e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:38.948403 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b6dfd-4f74-4fba-a364-aa77aed72e57 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:38.949057 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b6dfd-4f74-4fba-a364-aa77aed72e57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:38.949359 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b6dfd-4f74-4fba-a364-aa77aed72e57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:38.951388 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b6dfd-4f74-4fba-a364-aa77aed72e57 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:38.959174 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 215/445] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:38.959950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b6dfd-4f74-4fba-a364-aa77aed72e57 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:38.995373 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f14922-184f-4d52-87e7-200595d15dce None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:39.032557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71ae12-a563-43f9-8927-46dfbca9798e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:39.033166 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71ae12-a563-43f9-8927-46dfbca9798e None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:39.033443 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71ae12-a563-43f9-8927-46dfbca9798e None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:39.033713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71ae12-a563-43f9-8927-46dfbca9798e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:39.034488 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71ae12-a563-43f9-8927-46dfbca9798e None admin] RBAC: Adding query filter params (group.id=dd13637587354a10893278b6d62d73ca, role.id=cc8bd97c7b9745d1a52746650a2e49b9, scope.project.id=a8b13074ade34e4d944896277be959f5) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:39.034749 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71ae12-a563-43f9-8927-46dfbca9798e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:39.036222 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71ae12-a563-43f9-8927-46dfbca9798e None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:39.042055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 231/446] 10.222.0.67 () {56 vars in 1376 bytes} [Wed Aug 31 00:07:38 2022] GET /identity/v3/role_assignments?group.id=dd13637587354a10893278b6d62d73ca&scope.project.id=a8b13074ade34e4d944896277be959f5&role.id=cc8bd97c7b9745d1a52746650a2e49b9 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:39.042831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71ae12-a563-43f9-8927-46dfbca9798e None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:39.063888 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b6dfd-4f74-4fba-a364-aa77aed72e57 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:39.072753 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:39.105017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40325ead-16a0-4738-9ac0-1f8da92bbff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:39.105694 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40325ead-16a0-4738-9ac0-1f8da92bbff6 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:39.105971 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40325ead-16a0-4738-9ac0-1f8da92bbff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:39.106238 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40325ead-16a0-4738-9ac0-1f8da92bbff6 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:39.107691 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40325ead-16a0-4738-9ac0-1f8da92bbff6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:39.107999 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40325ead-16a0-4738-9ac0-1f8da92bbff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:39.108955 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:39.109889 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40325ead-16a0-4738-9ac0-1f8da92bbff6 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:39.125324 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 216/447] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:07:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:39.126596 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40325ead-16a0-4738-9ac0-1f8da92bbff6 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:39.179198 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71ae12-a563-43f9-8927-46dfbca9798e None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:39.223798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ccfc6a-1a54-41ec-aa64-3d3cadb33a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:39.224407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ccfc6a-1a54-41ec-aa64-3d3cadb33a94 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:39.224695 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ccfc6a-1a54-41ec-aa64-3d3cadb33a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:39.224972 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ccfc6a-1a54-41ec-aa64-3d3cadb33a94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:39.250608 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40325ead-16a0-4738-9ac0-1f8da92bbff6 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:39.293623 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6814150-6ea2-483a-babd-12fa15734a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:39.294778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6814150-6ea2-483a-babd-12fa15734a3f None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:39.295138 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6814150-6ea2-483a-babd-12fa15734a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:39.295456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6814150-6ea2-483a-babd-12fa15734a3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:39.313150 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ccfc6a-1a54-41ec-aa64-3d3cadb33a94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:39.314810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ccfc6a-1a54-41ec-aa64-3d3cadb33a94 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:39.342301 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:39.368070 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:39.389257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6814150-6ea2-483a-babd-12fa15734a3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:39.390706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6814150-6ea2-483a-babd-12fa15734a3f None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:39.429651 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 232/448] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:39.436119 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ccfc6a-1a54-41ec-aa64-3d3cadb33a94 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:39.492395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 217/449] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:39.498245 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6814150-6ea2-483a-babd-12fa15734a3f None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:39.537946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ccfc6a-1a54-41ec-aa64-3d3cadb33a94 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:39.565236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:39.573472 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5eaba4-2b44-4fe6-a56d-2ac4681e7ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:39.574109 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5eaba4-2b44-4fe6-a56d-2ac4681e7ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:39.574384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5eaba4-2b44-4fe6-a56d-2ac4681e7ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:39.574654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5eaba4-2b44-4fe6-a56d-2ac4681e7ba6 None admin] PATH_INFO: `/v3/projects/1509e8ef0c6042559698c40a7a07a646` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:39.575865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5eaba4-2b44-4fe6-a56d-2ac4681e7ba6 None admin] RBAC: Authorizing `identity:get_project(project_id=1509e8ef0c6042559698c40a7a07a646)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:39.577304 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5eaba4-2b44-4fe6-a56d-2ac4681e7ba6 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:39.578932 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 233/450] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:39 2022] GET /identity/v3/projects/1509e8ef0c6042559698c40a7a07a646 => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:39.584304 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5eaba4-2b44-4fe6-a56d-2ac4681e7ba6 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:39.596510 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:39.600547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6814150-6ea2-483a-babd-12fa15734a3f None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:39.635120 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fe1cee-fe29-49de-a22d-c068f947fa96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:39.635749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe1cee-fe29-49de-a22d-c068f947fa96 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:39.636025 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe1cee-fe29-49de-a22d-c068f947fa96 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:39.636297 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe1cee-fe29-49de-a22d-c068f947fa96 None admin] PATH_INFO: `/v3/projects/a8b13074ade34e4d944896277be959f5` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:39.637412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fe1cee-fe29-49de-a22d-c068f947fa96 None admin] RBAC: Authorizing `identity:get_project(project_id=a8b13074ade34e4d944896277be959f5)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:39.638953 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fe1cee-fe29-49de-a22d-c068f947fa96 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:39.640591 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 218/451] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:07:39 2022] GET /identity/v3/projects/a8b13074ade34e4d944896277be959f5 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:39.645968 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fe1cee-fe29-49de-a22d-c068f947fa96 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:39.687895 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5eaba4-2b44-4fe6-a56d-2ac4681e7ba6 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:39.722761 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa508ce-895d-48e7-826d-2a767d1b74dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:39.723548 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa508ce-895d-48e7-826d-2a767d1b74dd None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:39.723689 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa508ce-895d-48e7-826d-2a767d1b74dd None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:39.723987 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa508ce-895d-48e7-826d-2a767d1b74dd None admin] PATH_INFO: `/v3/groups/3a94bbd0a8d746a18b250f92578f74a8` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:39.725048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa508ce-895d-48e7-826d-2a767d1b74dd None admin] RBAC: Authorizing `identity:get_group(group_id=3a94bbd0a8d746a18b250f92578f74a8)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:39.726491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa508ce-895d-48e7-826d-2a767d1b74dd None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:39.728066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 234/452] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 31 00:07:39 2022] GET /identity/v3/groups/3a94bbd0a8d746a18b250f92578f74a8 => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:39.734406 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa508ce-895d-48e7-826d-2a767d1b74dd None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:39.748959 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fe1cee-fe29-49de-a22d-c068f947fa96 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:39.782589 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa9b6b-b517-45e5-868b-dbc037de3c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:39.783242 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa9b6b-b517-45e5-868b-dbc037de3c4e None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:39.783579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa9b6b-b517-45e5-868b-dbc037de3c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:39.783852 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa9b6b-b517-45e5-868b-dbc037de3c4e None admin] PATH_INFO: `/v3/groups/dd13637587354a10893278b6d62d73ca` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:39.784947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa9b6b-b517-45e5-868b-dbc037de3c4e None admin] RBAC: Authorizing `identity:get_group(group_id=dd13637587354a10893278b6d62d73ca)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:39.786394 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa9b6b-b517-45e5-868b-dbc037de3c4e None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:39.788268 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 219/453] 10.222.0.67 () {56 vars in 1180 bytes} [Wed Aug 31 00:07:39 2022] GET /identity/v3/groups/dd13637587354a10893278b6d62d73ca => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:39.794185 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa9b6b-b517-45e5-868b-dbc037de3c4e None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:39.797748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:39.813103 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:39.839785 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa508ce-895d-48e7-826d-2a767d1b74dd None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:39.874079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147350fb-7ab9-4f59-a8d3-9dcb3f8884d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:39.874679 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147350fb-7ab9-4f59-a8d3-9dcb3f8884d0 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:39.874955 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147350fb-7ab9-4f59-a8d3-9dcb3f8884d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:39.875501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147350fb-7ab9-4f59-a8d3-9dcb3f8884d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:39.876223 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147350fb-7ab9-4f59-a8d3-9dcb3f8884d0 None admin] RBAC: Adding query filter params (group.id=3a94bbd0a8d746a18b250f92578f74a8, role.id=34df06ce84ad4cf8b3bd1a4490228017, scope.project.id=1509e8ef0c6042559698c40a7a07a646) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:39.876530 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147350fb-7ab9-4f59-a8d3-9dcb3f8884d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:39.877966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147350fb-7ab9-4f59-a8d3-9dcb3f8884d0 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:39.883569 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 235/454] 10.222.0.67 () {56 vars in 1376 bytes} [Wed Aug 31 00:07:39 2022] GET /identity/v3/role_assignments?group.id=3a94bbd0a8d746a18b250f92578f74a8&scope.project.id=1509e8ef0c6042559698c40a7a07a646&role.id=34df06ce84ad4cf8b3bd1a4490228017 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:39.902045 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa9b6b-b517-45e5-868b-dbc037de3c4e None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:39.939776 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ba2b8-c39f-4512-b8e7-e6ed9933ffad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:39.940391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ba2b8-c39f-4512-b8e7-e6ed9933ffad None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:39.940667 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ba2b8-c39f-4512-b8e7-e6ed9933ffad None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:39.940935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ba2b8-c39f-4512-b8e7-e6ed9933ffad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:39.941676 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ba2b8-c39f-4512-b8e7-e6ed9933ffad None admin] RBAC: Adding query filter params (group.id=dd13637587354a10893278b6d62d73ca, role.id=4d91c00f40304b9e9ef4e6c89f46e5c5, scope.project.id=a8b13074ade34e4d944896277be959f5) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:39.941983 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ba2b8-c39f-4512-b8e7-e6ed9933ffad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:39.943476 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ba2b8-c39f-4512-b8e7-e6ed9933ffad None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:39.963073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 220/455] 10.222.0.67 () {56 vars in 1376 bytes} [Wed Aug 31 00:07:39 2022] GET /identity/v3/role_assignments?group.id=dd13637587354a10893278b6d62d73ca&scope.project.id=a8b13074ade34e4d944896277be959f5&role.id=4d91c00f40304b9e9ef4e6c89f46e5c5 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:42.493094 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5a0b3-5f07-4abb-a08e-24ec7da9910b None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:42.493620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5a0b3-5f07-4abb-a08e-24ec7da9910b None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:42.493620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5a0b3-5f07-4abb-a08e-24ec7da9910b None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:42.495404 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 236/456] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:42.503001 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85561532-1187-4778-a374-62c875e3996d None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:42.503971 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85561532-1187-4778-a374-62c875e3996d None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:42.504504 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85561532-1187-4778-a374-62c875e3996d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:42.525682 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-85561532-1187-4778-a374-62c875e3996d None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:42.527220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-85561532-1187-4778-a374-62c875e3996d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:42.574285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f1c6b-8b53-48fd-bb1e-acaa08672aca None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:42.574843 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f1c6b-8b53-48fd-bb1e-acaa08672aca None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:42.576106 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f1c6b-8b53-48fd-bb1e-acaa08672aca None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:42.578145 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 237/457] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:42.582851 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4f51f-d405-4307-b66d-295fe572190a None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:42.583391 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4f51f-d405-4307-b66d-295fe572190a None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:42.583842 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4f51f-d405-4307-b66d-295fe572190a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:42.612578 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-1ed4f51f-d405-4307-b66d-295fe572190a None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:42.614075 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-1ed4f51f-d405-4307-b66d-295fe572190a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:42.637309 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:42.650142 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 221/458] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:42.727228 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 238/459] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:42.930859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccfacab-26df-49b4-8832-8d3c85d31a38 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:42.931469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccfacab-26df-49b4-8832-8d3c85d31a38 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:42.931952 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccfacab-26df-49b4-8832-8d3c85d31a38 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:42.933474 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 222/460] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:42.938919 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044f5f3-a08b-4915-8844-d8209825990c None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:42.939446 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044f5f3-a08b-4915-8844-d8209825990c None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:42.939901 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044f5f3-a08b-4915-8844-d8209825990c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:42.959413 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-6044f5f3-a08b-4915-8844-d8209825990c None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:42.960936 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-6044f5f3-a08b-4915-8844-d8209825990c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:43.033292 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccfacab-26df-49b4-8832-8d3c85d31a38 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:43.046753 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 239/461] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:43.077361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6044f5f3-a08b-4915-8844-d8209825990c None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:43.082272 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:43.227074 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccfacab-26df-49b4-8832-8d3c85d31a38 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:43.292861 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6044f5f3-a08b-4915-8844-d8209825990c None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:43.305640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d168b-625a-4fb0-bfc5-f9e68be2f3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:43.306282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d168b-625a-4fb0-bfc5-f9e68be2f3c4 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:43.306557 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d168b-625a-4fb0-bfc5-f9e68be2f3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:43.306826 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d168b-625a-4fb0-bfc5-f9e68be2f3c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:43.326376 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d168b-625a-4fb0-bfc5-f9e68be2f3c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:43.327871 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d168b-625a-4fb0-bfc5-f9e68be2f3c4 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:43.361813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-8c2d168b-625a-4fb0-bfc5-f9e68be2f3c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:43.361813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:43.361813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:43.361813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:43.361813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:43.361813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:43.361813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:43.361813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:43.361813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:43.361813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:43.361813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:43.361813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:43.361813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:07:43.361813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:07:43.361813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:07:43.361813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:07:43.362770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:43.362770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:43.362770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:43.362770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:43.362770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:43.362770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:43.362770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:43.362770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:43.362770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:43.362770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:43.362770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:43.362770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:43.362770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:43.362770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:43.362770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:43.362770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:43.362770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:07:43.363632 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:07:43.363632 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:43.363632 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:43.363816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093b933-4d43-45ee-b76f-91227845a9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:43.363816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093b933-4d43-45ee-b76f-91227845a9c1 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:43.363816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093b933-4d43-45ee-b76f-91227845a9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:43.363816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093b933-4d43-45ee-b76f-91227845a9c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:43.366015 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 223/462] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:43.366994 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d168b-625a-4fb0-bfc5-f9e68be2f3c4 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:43.383498 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093b933-4d43-45ee-b76f-91227845a9c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:43.385354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093b933-4d43-45ee-b76f-91227845a9c1 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:43.407711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-6093b933-4d43-45ee-b76f-91227845a9c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:43.407711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:43.407711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:43.407711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:43.407711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:43.407711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:43.407711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:43.407711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:43.407711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:43.407711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:43.407711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:43.407711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:43.407711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:07:43.407711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:07:43.407711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:07:43.407711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:07:43.408665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:43.408665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:43.408665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:43.408665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:43.408665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:43.408665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:43.408665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:43.408665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:43.408665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:43.408665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:43.408665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:43.408665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:43.408665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:43.408665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:43.408665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:43.408665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:43.408665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:07:43.409499 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:07:43.409499 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:43.409499 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:43.409499 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 240/463] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:07:43.414665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093b933-4d43-45ee-b76f-91227845a9c1 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:43.433236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:43.482951 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:43.498649 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d168b-625a-4fb0-bfc5-f9e68be2f3c4 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:43.535094 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cff5dc-3527-4b37-bf5e-505e88ab1c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:43.535717 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093b933-4d43-45ee-b76f-91227845a9c1 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:43.536081 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cff5dc-3527-4b37-bf5e-505e88ab1c6e None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:43.536081 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cff5dc-3527-4b37-bf5e-505e88ab1c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:43.536308 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cff5dc-3527-4b37-bf5e-505e88ab1c6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:43.548023 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cff5dc-3527-4b37-bf5e-505e88ab1c6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:43.549495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cff5dc-3527-4b37-bf5e-505e88ab1c6e None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:43.564423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-b1cff5dc-3527-4b37-bf5e-505e88ab1c6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:43.564423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:43.564423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:43.564423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:43.564423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:43.564423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:43.564423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:43.564423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:43.564423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:43.564423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:43.564423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:43.564423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:43.564423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:07:43.564423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:07:43.564423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:07:43.564423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:07:43.565332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:43.565332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:43.565332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:43.565332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:43.565332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:43.565332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:43.565332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:43.565332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:43.565332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:43.565332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:43.565332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:43.565332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:43.565332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:43.565332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:43.565332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:43.565332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:43.565332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:07:43.566181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:07:43.566181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:43.566181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:43.566181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 224/464] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:43.566399 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cff5dc-3527-4b37-bf5e-505e88ab1c6e None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:43.578947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f79c10-551d-4910-9a21-2d3f7f9cc210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:43.579599 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f79c10-551d-4910-9a21-2d3f7f9cc210 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:43.579881 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f79c10-551d-4910-9a21-2d3f7f9cc210 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:43.580159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f79c10-551d-4910-9a21-2d3f7f9cc210 None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:43.580808 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f79c10-551d-4910-9a21-2d3f7f9cc210 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:43.581117 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f79c10-551d-4910-9a21-2d3f7f9cc210 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:43.582507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f79c10-551d-4910-9a21-2d3f7f9cc210 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:43.600516 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 241/465] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:07:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:43.600516 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a9421-1522-43e3-865a-b47604d68373 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:43.600516 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a9421-1522-43e3-865a-b47604d68373 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:43.600516 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a9421-1522-43e3-865a-b47604d68373 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:43.600516 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 242/466] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:43.600516 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a9421-1522-43e3-865a-b47604d68373 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:43.684142 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cff5dc-3527-4b37-bf5e-505e88ab1c6e None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:43.686633 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:43.713359 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a9421-1522-43e3-865a-b47604d68373 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:43.719587 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707ad0a-edff-43ff-80b5-6d2f1713d2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:43.720205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707ad0a-edff-43ff-80b5-6d2f1713d2d1 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:43.720484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707ad0a-edff-43ff-80b5-6d2f1713d2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:43.720757 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707ad0a-edff-43ff-80b5-6d2f1713d2d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:43.721404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707ad0a-edff-43ff-80b5-6d2f1713d2d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:43.721710 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707ad0a-edff-43ff-80b5-6d2f1713d2d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:43.723301 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707ad0a-edff-43ff-80b5-6d2f1713d2d1 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:43.729253 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:43.736025 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 225/467] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:07:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:43.740186 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707ad0a-edff-43ff-80b5-6d2f1713d2d1 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:43.753197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925ddc6-2083-4af2-a21c-305e288a8327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:43.753925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925ddc6-2083-4af2-a21c-305e288a8327 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:43.754200 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925ddc6-2083-4af2-a21c-305e288a8327 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:43.754470 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925ddc6-2083-4af2-a21c-305e288a8327 None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:43.755139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e925ddc6-2083-4af2-a21c-305e288a8327 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:43.755442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e925ddc6-2083-4af2-a21c-305e288a8327 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:43.756943 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e925ddc6-2083-4af2-a21c-305e288a8327 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:43.771111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 243/468] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:07:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:43.772596 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30073792-4a1d-4c36-9695-2c388c044a59 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:43.772992 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30073792-4a1d-4c36-9695-2c388c044a59 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:43.773418 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30073792-4a1d-4c36-9695-2c388c044a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:43.797220 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-30073792-4a1d-4c36-9695-2c388c044a59 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:43.798439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-30073792-4a1d-4c36-9695-2c388c044a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:43.854671 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707ad0a-edff-43ff-80b5-6d2f1713d2d1 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:43.875252 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 244/469] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:43.875252 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30073792-4a1d-4c36-9695-2c388c044a59 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:43.900981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c23ca-fad8-4329-8a76-58b3d82f1059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:43.901593 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c23ca-fad8-4329-8a76-58b3d82f1059 None admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:43.901874 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c23ca-fad8-4329-8a76-58b3d82f1059 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:43.902208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c23ca-fad8-4329-8a76-58b3d82f1059 None admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:43.902865 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c23ca-fad8-4329-8a76-58b3d82f1059 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:43.904842 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c23ca-fad8-4329-8a76-58b3d82f1059 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:43.939549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 226/470] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 31 00:07:43 2022] POST /identity/v3/users => generated 246 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:07:43.940655 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c23ca-fad8-4329-8a76-58b3d82f1059 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:43.975625 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:43.991183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30073792-4a1d-4c36-9695-2c388c044a59 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:43.999081 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:44.037088 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c705d1d-4d5b-40cb-b1a9-929d9ae70ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:44.037781 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c705d1d-4d5b-40cb-b1a9-929d9ae70ac4 None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:44.038062 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c705d1d-4d5b-40cb-b1a9-929d9ae70ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:44.038329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c705d1d-4d5b-40cb-b1a9-929d9ae70ac4 None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:44.038986 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c705d1d-4d5b-40cb-b1a9-929d9ae70ac4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:44.040520 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c705d1d-4d5b-40cb-b1a9-929d9ae70ac4 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:44.066667 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 245/471] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 31 00:07:43 2022] POST /identity/v3/users => generated 248 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:07:44.068146 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5020e1-7993-41fb-acf8-d340210454ad None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:44.068432 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5020e1-7993-41fb-acf8-d340210454ad None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:44.068757 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5020e1-7993-41fb-acf8-d340210454ad None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:44.070355 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 246/472] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:44.076802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0228b-2aaf-4241-825a-22b509e23135 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:44.077115 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0228b-2aaf-4241-825a-22b509e23135 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:44.077346 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0228b-2aaf-4241-825a-22b509e23135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:44.098806 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-0fe0228b-2aaf-4241-825a-22b509e23135 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:44.100174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-0fe0228b-2aaf-4241-825a-22b509e23135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:44.107301 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c23ca-fad8-4329-8a76-58b3d82f1059 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:44.149336 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59853893-3f24-4a38-b647-fcd05e232465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:44.150054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59853893-3f24-4a38-b647-fcd05e232465 None admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:44.150341 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59853893-3f24-4a38-b647-fcd05e232465 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:44.150612 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59853893-3f24-4a38-b647-fcd05e232465 None admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:44.151418 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59853893-3f24-4a38-b647-fcd05e232465 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:44.152880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59853893-3f24-4a38-b647-fcd05e232465 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:44.179907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 227/473] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 31 00:07:43 2022] POST /identity/v3/users => generated 248 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:07:44.187479 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 247/474] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:07:44.228815 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59853893-3f24-4a38-b647-fcd05e232465 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:44.328552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:44.362941 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59853893-3f24-4a38-b647-fcd05e232465 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:44.403778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3e737-efec-4130-8e00-301343e0c236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:44.404503 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3e737-efec-4130-8e00-301343e0c236 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:44.404781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3e737-efec-4130-8e00-301343e0c236 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:44.405054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3e737-efec-4130-8e00-301343e0c236 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:44.416554 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3e737-efec-4130-8e00-301343e0c236 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:44.418011 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3e737-efec-4130-8e00-301343e0c236 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:44.434068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-5ea3e737-efec-4130-8e00-301343e0c236 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:44.434068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:44.434068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:44.434068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:44.434068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:44.434068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:44.434068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:44.434068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:44.434068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:44.434068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:44.434068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:44.434068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:44.434068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:07:44.434068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:07:44.434068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:07:44.434068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:07:44.436198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:44.436198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:44.436198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:44.436198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:44.436198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:44.436198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:44.436198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:44.436198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:44.436198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:44.436198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:44.436198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:44.436198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:44.436198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:44.436198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:44.436198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:44.436198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:44.436198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:07:44.437102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:07:44.437102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:44.437102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:44.437102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 228/475] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:44.442212 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0228b-2aaf-4241-825a-22b509e23135 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:44.493481 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3e737-efec-4130-8e00-301343e0c236 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:44.541719 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:44.553332 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0228b-2aaf-4241-825a-22b509e23135 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:44.598563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd900c29-385f-4b20-ab41-608d95a2ece3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:44.598563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd900c29-385f-4b20-ab41-608d95a2ece3 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:44.599083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd900c29-385f-4b20-ab41-608d95a2ece3 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:44.599516 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd900c29-385f-4b20-ab41-608d95a2ece3 None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:44.600266 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd900c29-385f-4b20-ab41-608d95a2ece3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:44.600671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd900c29-385f-4b20-ab41-608d95a2ece3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:44.602277 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd900c29-385f-4b20-ab41-608d95a2ece3 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:44.616908 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 248/476] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:44.622675 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd900c29-385f-4b20-ab41-608d95a2ece3 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:44.623110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3e737-efec-4130-8e00-301343e0c236 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:44.662744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:44.680097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161ae39-10f4-4247-8ef6-220574314231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:44.680890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161ae39-10f4-4247-8ef6-220574314231 None admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:44.681271 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161ae39-10f4-4247-8ef6-220574314231 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:44.681632 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161ae39-10f4-4247-8ef6-220574314231 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:44.682407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0161ae39-10f4-4247-8ef6-220574314231 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:44.685239 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0161ae39-10f4-4247-8ef6-220574314231 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:44.697025 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.sql.core [None req-0161ae39-10f4-4247-8ef6-220574314231 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 00:07:44.697025 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 00:07:44.697025 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [parameters: {'id': 'bc646600d34f46bea2dc73dc806920c6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 00:07:44.697025 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81471) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 00:07:44.719124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-0161ae39-10f4-4247-8ef6-220574314231 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 00:07:44.719124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:44.719124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 00:07:44.719124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 00:07:44.719124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 00:07:44.719124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 00:07:44.719124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 00:07:44.719124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 00:07:44.719124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 00:07:44.719124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application conn.query(q) Aug 31 00:07:44.719124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 00:07:44.719124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 00:07:44.719124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 00:07:44.719124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application result.read() Aug 31 00:07:44.719124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 00:07:44.720221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 00:07:44.720221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 00:07:44.720221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 00:07:44.720221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 00:07:44.720221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 00:07:44.720221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 00:07:44.720221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 00:07:44.720221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 00:07:44.720221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:44.720221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 00:07:44.720221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:44.720221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:44.720221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 00:07:44.720221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 00:07:44.720221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 31 00:07:44.720221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return ref.to_dict() Aug 31 00:07:44.720221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 00:07:44.720221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application next(self.gen) Aug 31 00:07:44.721927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 00:07:44.721927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application yield resource Aug 31 00:07:44.721927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 00:07:44.721927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application next(self.gen) Aug 31 00:07:44.721927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 00:07:44.721927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 00:07:44.721927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 00:07:44.721927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 00:07:44.721927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 00:07:44.721927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise self.value Aug 31 00:07:44.721927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 00:07:44.721927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 00:07:44.721927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 00:07:44.721927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application session.commit() Aug 31 00:07:44.721927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 00:07:44.721927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 00:07:44.721927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 00:07:44.723130 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 00:07:44.723130 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 00:07:44.723130 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self.session.flush() Aug 31 00:07:44.723130 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 00:07:44.723130 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 00:07:44.723130 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 00:07:44.723130 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 00:07:44.723130 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 00:07:44.723130 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application compat.raise_( Aug 31 00:07:44.723130 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 00:07:44.723130 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception Aug 31 00:07:44.723130 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 00:07:44.723130 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 00:07:44.723130 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 00:07:44.723130 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 00:07:44.723130 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 00:07:44.723130 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 00:07:44.724577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 00:07:44.724577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 00:07:44.724577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 00:07:44.724577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 00:07:44.724577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 00:07:44.724577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 00:07:44.724577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 00:07:44.724577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 00:07:44.724577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 00:07:44.724577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 00:07:44.724577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 00:07:44.724577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 00:07:44.724577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 00:07:44.724577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 00:07:44.724577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 00:07:44.724577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception Aug 31 00:07:44.726701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 00:07:44.726701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 00:07:44.726701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 00:07:44.726701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 00:07:44.726701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 00:07:44.726701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 00:07:44.726701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 00:07:44.726701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application conn.query(q) Aug 31 00:07:44.726701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 00:07:44.726701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 00:07:44.726701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 00:07:44.726701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application result.read() Aug 31 00:07:44.726701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 00:07:44.726701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 00:07:44.726701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 00:07:44.726701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 00:07:44.728974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 00:07:44.728974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 00:07:44.728974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 00:07:44.728974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 00:07:44.728974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 00:07:44.728974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 00:07:44.728974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [parameters: {'id': 'bc646600d34f46bea2dc73dc806920c6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 00:07:44.728974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 00:07:44.728974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:44.728974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:07:44.728974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:44.728974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:44.728974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:44.728974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:44.728974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:44.728974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:44.730972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:44.730972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:44.730972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:44.730972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:44.730972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:44.730972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:44.730972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 31 00:07:44.730972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 31 00:07:44.730972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:44.730972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:44.730972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 31 00:07:44.730972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 31 00:07:44.730972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 00:07:44.730972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 00:07:44.730972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 00:07:44.730972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:44.733118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 229/477] 10.222.0.67 () {60 vars in 1131 bytes} [Wed Aug 31 00:07:44 2022] POST /identity/v3/roles => generated 144 bytes in 228 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:07:44.733118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161ae39-10f4-4247-8ef6-220574314231 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:44.772598 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd900c29-385f-4b20-ab41-608d95a2ece3 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:44.818619 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be759f8-5e81-4889-a1d9-449f1fe9c0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:44.819672 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be759f8-5e81-4889-a1d9-449f1fe9c0ab None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:44.820210 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be759f8-5e81-4889-a1d9-449f1fe9c0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:44.820669 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be759f8-5e81-4889-a1d9-449f1fe9c0ab None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:44.822772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be759f8-5e81-4889-a1d9-449f1fe9c0ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:44.825781 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be759f8-5e81-4889-a1d9-449f1fe9c0ab None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:44.876421 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 249/478] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 31 00:07:44 2022] POST /identity/v3/users => generated 249 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:07:44.917899 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161ae39-10f4-4247-8ef6-220574314231 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:44.957122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2053c8-2b54-4538-94f7-74b80be5b43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:44.958121 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2053c8-2b54-4538-94f7-74b80be5b43a None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:44.958647 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2053c8-2b54-4538-94f7-74b80be5b43a None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:44.959152 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2053c8-2b54-4538-94f7-74b80be5b43a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:44.972060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2053c8-2b54-4538-94f7-74b80be5b43a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:44.977766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-cc2053c8-2b54-4538-94f7-74b80be5b43a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:07:44.977766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:44.977766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:44.977766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:44.977766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:44.977766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:44.977766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:44.977766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:44.977766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:44.977766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:44.977766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:44.977766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:44.977766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:44.977766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:44.977766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:44.977766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:44.978631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:44.978631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:44.978631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:44.978631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:44.978631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:44.978631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:44.978631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:44.978631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:44.978631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:44.978631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:44.978631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:44.978631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:44.978631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:44.978631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:44.978631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:44.978631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:44.979440 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:44.979440 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:44.979440 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:44.979440 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:44.979440 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:44.979440 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:44.979440 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:44.979440 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:44.979440 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:44.979440 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:44.979440 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 00:07:44.979440 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:44.982336 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 230/479] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 31 00:07:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:44.988917 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be759f8-5e81-4889-a1d9-449f1fe9c0ab None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:45.009277 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:45.110608 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be759f8-5e81-4889-a1d9-449f1fe9c0ab None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:45.147295 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a4d61-bca5-4a8c-b615-0e145d19f16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:45.147831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a4d61-bca5-4a8c-b615-0e145d19f16f None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:45.148107 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a4d61-bca5-4a8c-b615-0e145d19f16f None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:45.148376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a4d61-bca5-4a8c-b615-0e145d19f16f None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:45.149035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a4d61-bca5-4a8c-b615-0e145d19f16f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:45.149333 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a4d61-bca5-4a8c-b615-0e145d19f16f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:45.150784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a4d61-bca5-4a8c-b615-0e145d19f16f None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:45.159770 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 250/480] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:07:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:46.572821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72186867-660f-45ad-90e2-a169e172730c None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:46.573330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72186867-660f-45ad-90e2-a169e172730c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:46.573404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72186867-660f-45ad-90e2-a169e172730c None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:46.574998 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 231/481] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:46.580622 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9d9ac-4a9c-4300-947a-137b22e2d013 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:46.581056 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9d9ac-4a9c-4300-947a-137b22e2d013 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:46.581240 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9d9ac-4a9c-4300-947a-137b22e2d013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:46.608214 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-55e9d9ac-4a9c-4300-947a-137b22e2d013 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:46.609431 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-55e9d9ac-4a9c-4300-947a-137b22e2d013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:46.690824 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:46.696581 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 251/482] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:46.717758 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ffd06-0ff7-48b0-8954-9a320e673c5c None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:46.718015 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ffd06-0ff7-48b0-8954-9a320e673c5c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:46.718296 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ffd06-0ff7-48b0-8954-9a320e673c5c None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:46.719986 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 232/483] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:46.725088 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee93672-36e5-43be-9cbb-02bdafb146a9 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:46.725380 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee93672-36e5-43be-9cbb-02bdafb146a9 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:46.725656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee93672-36e5-43be-9cbb-02bdafb146a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:46.752324 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-eee93672-36e5-43be-9cbb-02bdafb146a9 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:46.753567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-eee93672-36e5-43be-9cbb-02bdafb146a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:46.848704 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 252/484] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:46.987206 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb3fe4-8da1-4748-b668-e34b1543a10c None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:46.987821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb3fe4-8da1-4748-b668-e34b1543a10c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:46.988295 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb3fe4-8da1-4748-b668-e34b1543a10c None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:46.995064 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651b2b4-98ba-43ef-b7c7-a5d6a1ebe44e None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:46.995274 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651b2b4-98ba-43ef-b7c7-a5d6a1ebe44e None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:46.995553 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651b2b4-98ba-43ef-b7c7-a5d6a1ebe44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:46.995907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 233/485] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:46 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:47.024147 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-6651b2b4-98ba-43ef-b7c7-a5d6a1ebe44e None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:47.025351 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-6651b2b4-98ba-43ef-b7c7-a5d6a1ebe44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:47.092146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcb3fe4-8da1-4748-b668-e34b1543a10c None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:47.128686 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 253/486] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:07:47.194478 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:47.205473 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651b2b4-98ba-43ef-b7c7-a5d6a1ebe44e None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:47.284552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcb3fe4-8da1-4748-b668-e34b1543a10c None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:47.309152 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:47.354061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65be87-6769-403e-b41a-361ab946a070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:47.354710 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65be87-6769-403e-b41a-361ab946a070 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:47.355829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65be87-6769-403e-b41a-361ab946a070 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:47.356108 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65be87-6769-403e-b41a-361ab946a070 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:47.375375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651b2b4-98ba-43ef-b7c7-a5d6a1ebe44e None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:47.376495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc65be87-6769-403e-b41a-361ab946a070 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:47.385520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-fc65be87-6769-403e-b41a-361ab946a070 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:07:47.385520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:47.385520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:47.385520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:47.385520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:47.385520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:47.385520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:47.385520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:47.385520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:47.385520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:47.385520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:47.385520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:47.385520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:47.385520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:47.385520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:47.385520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:47.386417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:47.386417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:47.386417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:47.386417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:47.386417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:47.386417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:47.386417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:47.386417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:47.386417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:47.386417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:47.386417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:47.386417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:47.386417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:47.386417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:47.386417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:47.386417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:47.387240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:47.387240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:47.387240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:47.387240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:47.387240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:47.387240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:47.387240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:47.387240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:47.387240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:47.387240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:47.387240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:07:47.387240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:47.387240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 234/487] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 31 00:07:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:47.394853 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65be87-6769-403e-b41a-361ab946a070 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:47.419298 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4597ad-1406-4e55-9162-b28b8bc7c40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:47.420342 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4597ad-1406-4e55-9162-b28b8bc7c40f None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:47.420843 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4597ad-1406-4e55-9162-b28b8bc7c40f None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:47.421314 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4597ad-1406-4e55-9162-b28b8bc7c40f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:47.433402 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4597ad-1406-4e55-9162-b28b8bc7c40f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:47.440655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-ef4597ad-1406-4e55-9162-b28b8bc7c40f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:07:47.440655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:47.440655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:47.440655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:47.440655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:47.440655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:47.440655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:47.440655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:47.440655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:47.440655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:47.440655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:47.440655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:47.440655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:47.440655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:47.440655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:47.440655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:47.441565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:47.441565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:47.441565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:47.441565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:47.441565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:47.441565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:47.441565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:47.441565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:47.441565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:47.441565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:47.441565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:47.441565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:47.441565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:47.441565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:47.441565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:47.441565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:47.442354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:47.442354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:47.442354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:47.442354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:47.442354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:47.442354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:47.442354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:47.442354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:47.442354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:47.442354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:47.442354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:07:47.442354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:47.445248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 254/488] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 31 00:07:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:47.456559 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4597ad-1406-4e55-9162-b28b8bc7c40f None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:47.540942 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65be87-6769-403e-b41a-361ab946a070 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:47.552442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:47.577752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8c69f-1e74-4008-b432-f4cb196b0c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:47.578368 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8c69f-1e74-4008-b432-f4cb196b0c9b None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:47.578624 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8c69f-1e74-4008-b432-f4cb196b0c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:47.578882 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8c69f-1e74-4008-b432-f4cb196b0c9b None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:47.579586 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8c69f-1e74-4008-b432-f4cb196b0c9b None admin] RBAC: Adding query filter params (name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:47.579879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8c69f-1e74-4008-b432-f4cb196b0c9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:47.581236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8c69f-1e74-4008-b432-f4cb196b0c9b None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:47.582860 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4597ad-1406-4e55-9162-b28b8bc7c40f None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:47.594305 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 235/489] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 31 00:07:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:07:47.594305 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8c69f-1e74-4008-b432-f4cb196b0c9b None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:47.613027 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:47.634443 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146386f6-de03-47e2-9fe4-d4429b950c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:47.635505 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146386f6-de03-47e2-9fe4-d4429b950c3d None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:47.636046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146386f6-de03-47e2-9fe4-d4429b950c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:47.636585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146386f6-de03-47e2-9fe4-d4429b950c3d None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:47.638133 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146386f6-de03-47e2-9fe4-d4429b950c3d None admin] RBAC: Adding query filter params (name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:47.638646 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146386f6-de03-47e2-9fe4-d4429b950c3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:47.640461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146386f6-de03-47e2-9fe4-d4429b950c3d None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:47.650058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 255/490] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 31 00:07:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:47.651176 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146386f6-de03-47e2-9fe4-d4429b950c3d None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:47.710767 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8c69f-1e74-4008-b432-f4cb196b0c9b None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:47.745928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34f24a-e5c8-483a-bc7f-c53d383e8428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:47.746544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34f24a-e5c8-483a-bc7f-c53d383e8428 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:47.747398 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34f24a-e5c8-483a-bc7f-c53d383e8428 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:47.747669 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34f24a-e5c8-483a-bc7f-c53d383e8428 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:47.758166 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34f24a-e5c8-483a-bc7f-c53d383e8428 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:47.764148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-af34f24a-e5c8-483a-bc7f-c53d383e8428 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:07:47.764148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:47.764148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:47.764148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:47.764148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:47.764148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:47.764148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:47.764148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:47.764148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:47.764148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:47.764148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:47.764148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:47.764148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:47.764148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:47.764148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:47.764148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:47.765150 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:47.765150 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:47.765150 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:47.765150 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:47.765150 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:47.765150 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:47.765150 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:47.765150 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:47.765150 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:47.765150 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:47.765150 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:47.765150 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:47.765150 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:47.765150 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:47.765150 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:47.765150 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:47.765944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:47.765944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:47.765944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:47.765944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:47.765944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:47.765944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:47.765944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:47.765944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:47.765944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:47.765944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:47.765944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:07:47.765944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:47.765944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 236/491] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 31 00:07:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:47.766598 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34f24a-e5c8-483a-bc7f-c53d383e8428 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:47.776119 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146386f6-de03-47e2-9fe4-d4429b950c3d None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:47.800900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:47.814344 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd584ec-2473-452c-a89f-3361612e97f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:47.814967 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd584ec-2473-452c-a89f-3361612e97f4 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:47.815351 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd584ec-2473-452c-a89f-3361612e97f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:47.815594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd584ec-2473-452c-a89f-3361612e97f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:47.827472 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd584ec-2473-452c-a89f-3361612e97f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:47.828929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd584ec-2473-452c-a89f-3361612e97f4 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:47.843706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-cfd584ec-2473-452c-a89f-3361612e97f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:47.843706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:47.843706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:47.843706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:47.843706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:47.843706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:47.843706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:47.843706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:47.843706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:47.843706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:47.843706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:47.843706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:47.843706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:07:47.843706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:07:47.843706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:07:47.843706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:07:47.844637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:47.844637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:47.844637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:47.844637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:47.844637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:47.844637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:47.844637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:47.844637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:47.844637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:47.844637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:47.844637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:47.844637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:47.844637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:47.844637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:47.844637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:47.844637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:47.844637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:07:47.850386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:07:47.850386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:47.850386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:47.850386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 256/492] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:47.850386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a9d7d-49e4-4c99-bba7-7235ee0d290f None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:47.850386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a9d7d-49e4-4c99-bba7-7235ee0d290f None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:47.850386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a9d7d-49e4-4c99-bba7-7235ee0d290f None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:47.852503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 257/493] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:47.853202 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a9d7d-49e4-4c99-bba7-7235ee0d290f None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:47.877796 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34f24a-e5c8-483a-bc7f-c53d383e8428 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:47.891142 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:47.914253 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f0cc90-9b9f-4bec-9b3b-7aa2579954d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:47.915023 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0cc90-9b9f-4bec-9b3b-7aa2579954d4 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:47.915220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0cc90-9b9f-4bec-9b3b-7aa2579954d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:47.915496 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0cc90-9b9f-4bec-9b3b-7aa2579954d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:47.926299 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f0cc90-9b9f-4bec-9b3b-7aa2579954d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:47.927755 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f0cc90-9b9f-4bec-9b3b-7aa2579954d4 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:47.942454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-99f0cc90-9b9f-4bec-9b3b-7aa2579954d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:47.942454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:47.942454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:47.942454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:47.942454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:47.942454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:47.942454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:47.942454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:47.942454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:47.942454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:47.942454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:47.942454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:47.942454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:07:47.942454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:07:47.942454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:07:47.942454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:07:47.943403 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:47.943403 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:47.943403 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:47.943403 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:47.943403 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:47.943403 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:47.943403 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:47.943403 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:47.943403 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:47.943403 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:47.943403 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:47.943403 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:47.943403 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:47.943403 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:47.943403 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:47.943403 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:47.943403 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:07:47.947768 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:07:47.947768 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:47.947768 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:47.947768 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 237/494] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:47.947768 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f0cc90-9b9f-4bec-9b3b-7aa2579954d4 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:47.972292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a9d7d-49e4-4c99-bba7-7235ee0d290f None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:48.006494 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd1080-76b6-481e-9a82-d89035a235d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:48.007128 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd1080-76b6-481e-9a82-d89035a235d9 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:48.007405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd1080-76b6-481e-9a82-d89035a235d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:48.007675 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd1080-76b6-481e-9a82-d89035a235d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:48.008327 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd1080-76b6-481e-9a82-d89035a235d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:48.008629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd1080-76b6-481e-9a82-d89035a235d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:48.009991 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd1080-76b6-481e-9a82-d89035a235d9 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:48.017809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 258/495] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 31 00:07:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:48.018962 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95a708-5177-4423-9ce9-9bd23f6466e5 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:48.019357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95a708-5177-4423-9ce9-9bd23f6466e5 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:48.019629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95a708-5177-4423-9ce9-9bd23f6466e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:48.033684 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:48.037482 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-5e95a708-5177-4423-9ce9-9bd23f6466e5 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:48.038657 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-5e95a708-5177-4423-9ce9-9bd23f6466e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:48.049520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f0cc90-9b9f-4bec-9b3b-7aa2579954d4 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:48.085989 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897b9fd-0993-4f2e-a000-242267140f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:48.086744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897b9fd-0993-4f2e-a000-242267140f00 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:48.086870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897b9fd-0993-4f2e-a000-242267140f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:48.087134 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897b9fd-0993-4f2e-a000-242267140f00 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:48.087774 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9897b9fd-0993-4f2e-a000-242267140f00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:48.088075 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9897b9fd-0993-4f2e-a000-242267140f00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:48.089427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9897b9fd-0993-4f2e-a000-242267140f00 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:48.101370 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 238/496] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:48.102521 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ca202-641a-4f67-b546-95f59b43011d None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:48.102800 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ca202-641a-4f67-b546-95f59b43011d None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:48.103090 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ca202-641a-4f67-b546-95f59b43011d None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:48.107129 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 239/497] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:48.107129 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ca202-641a-4f67-b546-95f59b43011d None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:48.111508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 259/498] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:48.112379 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e95a708-5177-4423-9ce9-9bd23f6466e5 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:48.147978 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:48.217736 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ca202-641a-4f67-b546-95f59b43011d None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:48.229817 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e95a708-5177-4423-9ce9-9bd23f6466e5 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:48.254459 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8558fdf-8899-4547-bbba-248badfd923b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:48.255205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8558fdf-8899-4547-bbba-248badfd923b None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:48.255487 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8558fdf-8899-4547-bbba-248badfd923b None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:48.255878 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8558fdf-8899-4547-bbba-248badfd923b None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:48.256518 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8558fdf-8899-4547-bbba-248badfd923b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:48.256822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8558fdf-8899-4547-bbba-248badfd923b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:48.258295 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8558fdf-8899-4547-bbba-248badfd923b None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:48.267265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d3675-0dc4-447a-8c6c-7929bb03cc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:48.267900 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d3675-0dc4-447a-8c6c-7929bb03cc51 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:48.268174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d3675-0dc4-447a-8c6c-7929bb03cc51 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:48.268445 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d3675-0dc4-447a-8c6c-7929bb03cc51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:48.275373 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 240/499] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:48.275373 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8558fdf-8899-4547-bbba-248badfd923b None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:48.279017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:48.284948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220d3675-0dc4-447a-8c6c-7929bb03cc51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:48.286707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220d3675-0dc4-447a-8c6c-7929bb03cc51 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:48.303375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-220d3675-0dc4-447a-8c6c-7929bb03cc51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:48.303375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:48.303375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:48.303375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:48.303375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:48.303375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:48.303375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:48.303375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:48.303375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:48.303375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:48.303375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:48.303375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:48.303375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:07:48.303375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:07:48.303375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:07:48.303375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:07:48.304299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:48.304299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:48.304299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:48.304299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:48.304299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:48.304299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:48.304299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:48.304299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:48.304299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:48.304299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:48.304299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:48.304299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:48.304299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:48.304299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:48.304299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:48.304299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:48.304299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:07:48.305130 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:07:48.305130 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:48.305130 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:48.305130 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 260/500] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:48.305823 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097ca85-ec15-4ae4-a078-1573a370c6b2 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:48.306040 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097ca85-ec15-4ae4-a078-1573a370c6b2 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:48.306323 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097ca85-ec15-4ae4-a078-1573a370c6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:48.326319 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-f097ca85-ec15-4ae4-a078-1573a370c6b2 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:48.327542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f097ca85-ec15-4ae4-a078-1573a370c6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:48.391881 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8558fdf-8899-4547-bbba-248badfd923b None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:48.398843 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 261/501] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:48.399726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f097ca85-ec15-4ae4-a078-1573a370c6b2 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:48.417534 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:48.427425 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c0227-5dfe-473c-9a6b-140d1eb9faf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:48.428056 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c0227-5dfe-473c-9a6b-140d1eb9faf0 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:48.428339 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c0227-5dfe-473c-9a6b-140d1eb9faf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:48.428614 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c0227-5dfe-473c-9a6b-140d1eb9faf0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:48.437067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637c0227-5dfe-473c-9a6b-140d1eb9faf0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:48.438527 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637c0227-5dfe-473c-9a6b-140d1eb9faf0 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:48.451434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-637c0227-5dfe-473c-9a6b-140d1eb9faf0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:07:48.451434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:48.451434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:48.451434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:48.451434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:48.451434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:48.451434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:48.451434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:48.451434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:48.451434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:48.451434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:48.451434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:48.451434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:07:48.451434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:07:48.451434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:07:48.451434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:07:48.452552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:48.452552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:48.452552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:07:48.452552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:07:48.452552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:07:48.452552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:07:48.452552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:48.452552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:48.452552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:48.452552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:48.452552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:48.452552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:48.452552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:48.452552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:48.452552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:48.452552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:48.452552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:48.452552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:48.453652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:48.453652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:48.453652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:07:48.453652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:07:48.453652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:07:48.453652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:07:48.453652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:07:48.453652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:07:48.453652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:07:48.453652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:48.453652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 241/502] 10.222.0.67 () {56 vars in 1093 bytes} [Wed Aug 31 00:07:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:48.453652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c0227-5dfe-473c-9a6b-140d1eb9faf0 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:48.525298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:48.531467 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f097ca85-ec15-4ae4-a078-1573a370c6b2 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:48.566990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c0227-5dfe-473c-9a6b-140d1eb9faf0 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:48.568719 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcfed10-d0aa-48a6-abeb-7052900ea34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:48.569348 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfed10-d0aa-48a6-abeb-7052900ea34d None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:48.569631 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfed10-d0aa-48a6-abeb-7052900ea34d None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:48.569903 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfed10-d0aa-48a6-abeb-7052900ea34d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:48.577101 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcfed10-d0aa-48a6-abeb-7052900ea34d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:48.578552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcfed10-d0aa-48a6-abeb-7052900ea34d None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:48.591705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-0dcfed10-d0aa-48a6-abeb-7052900ea34d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 00:07:48.591705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:48.591705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:48.591705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:48.591705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:48.591705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:48.591705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:48.591705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:48.591705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:48.591705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:48.591705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:48.591705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:48.591705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:07:48.591705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:07:48.591705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:07:48.591705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:07:48.593066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:48.593066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:48.593066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:07:48.593066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:07:48.593066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:07:48.593066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:07:48.593066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:48.593066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:48.593066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:48.593066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:48.593066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:48.593066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:48.593066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:48.593066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:48.593066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:48.593066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:48.593066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:48.593066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:48.593928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:48.593928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:48.593928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:07:48.593928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:07:48.593928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:07:48.593928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:07:48.593928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:07:48.593928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:07:48.593928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 00:07:48.593928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:48.593928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 262/503] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:48.593928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcfed10-d0aa-48a6-abeb-7052900ea34d None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:48.613583 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7b78f-3bba-459a-b6b2-9f269f30400c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:48.614883 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7b78f-3bba-459a-b6b2-9f269f30400c None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:48.615356 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7b78f-3bba-459a-b6b2-9f269f30400c None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:48.615798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7b78f-3bba-459a-b6b2-9f269f30400c None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:48.616746 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7b78f-3bba-459a-b6b2-9f269f30400c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:48.617266 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7b78f-3bba-459a-b6b2-9f269f30400c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:48.618795 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7b78f-3bba-459a-b6b2-9f269f30400c None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:48.636576 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 242/504] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:48.636576 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7b78f-3bba-459a-b6b2-9f269f30400c None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:48.673383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:48.709552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcfed10-d0aa-48a6-abeb-7052900ea34d None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:48.740056 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7b78f-3bba-459a-b6b2-9f269f30400c None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:48.742351 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f11ba-deb5-4976-b2be-3b4eaf8483c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:48.743083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f11ba-deb5-4976-b2be-3b4eaf8483c5 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:48.743382 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f11ba-deb5-4976-b2be-3b4eaf8483c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:48.743643 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f11ba-deb5-4976-b2be-3b4eaf8483c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:48.755234 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f11ba-deb5-4976-b2be-3b4eaf8483c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:48.760552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-9e6f11ba-deb5-4976-b2be-3b4eaf8483c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:07:48.760552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:48.760552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:48.760552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:48.760552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:48.760552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:48.760552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:48.760552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:48.760552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:48.760552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:48.760552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:48.760552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:48.760552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:48.760552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:48.760552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:48.760552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:48.761529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:48.761529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:48.761529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:48.761529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:48.761529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:48.761529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:48.761529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:48.761529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:48.761529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:48.761529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:48.761529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:48.761529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:48.761529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:48.761529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:48.761529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:48.761529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:48.762293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:48.762293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:48.762293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:48.762293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:48.762293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:48.762293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:48.762293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:48.762293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:48.762293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:48.762293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:48.762293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:07:48.762293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:48.762293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 263/505] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 31 00:07:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:48.763711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f11ba-deb5-4976-b2be-3b4eaf8483c5 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:48.765035 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:48.776451 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa3e03-d718-4d94-89a4-7b907a6b856d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:48.777060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa3e03-d718-4d94-89a4-7b907a6b856d None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:48.777335 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa3e03-d718-4d94-89a4-7b907a6b856d None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:48.777606 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa3e03-d718-4d94-89a4-7b907a6b856d None admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:48.778367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa3e03-d718-4d94-89a4-7b907a6b856d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:48.778666 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa3e03-d718-4d94-89a4-7b907a6b856d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:48.780138 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa3e03-d718-4d94-89a4-7b907a6b856d None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:48.816028 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 243/506] 10.222.0.67 () {56 vars in 1133 bytes} [Wed Aug 31 00:07:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:48.816848 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa3e03-d718-4d94-89a4-7b907a6b856d None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:48.868628 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f11ba-deb5-4976-b2be-3b4eaf8483c5 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:48.894067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:48.903018 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0425d95-ee6f-465d-9d95-9b1cfc048fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:48.904196 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0425d95-ee6f-465d-9d95-9b1cfc048fbe None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:48.904463 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0425d95-ee6f-465d-9d95-9b1cfc048fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:48.904732 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0425d95-ee6f-465d-9d95-9b1cfc048fbe None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:48.905456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0425d95-ee6f-465d-9d95-9b1cfc048fbe None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:48.905758 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0425d95-ee6f-465d-9d95-9b1cfc048fbe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:48.907215 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0425d95-ee6f-465d-9d95-9b1cfc048fbe None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:48.915322 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa3e03-d718-4d94-89a4-7b907a6b856d None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:48.946949 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 264/507] 10.222.0.67 () {56 vars in 1137 bytes} [Wed Aug 31 00:07:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:48.946949 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0425d95-ee6f-465d-9d95-9b1cfc048fbe None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:48.951164 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763272e-1b72-4416-8997-ff7544a11c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:48.951790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763272e-1b72-4416-8997-ff7544a11c9d None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:48.952067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763272e-1b72-4416-8997-ff7544a11c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:48.952335 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763272e-1b72-4416-8997-ff7544a11c9d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:48.959354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0763272e-1b72-4416-8997-ff7544a11c9d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:48.960789 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0763272e-1b72-4416-8997-ff7544a11c9d None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:48.972735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-0763272e-1b72-4416-8997-ff7544a11c9d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 00:07:48.972735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:48.972735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:48.972735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:48.972735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:48.972735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:48.972735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:48.972735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:48.972735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:48.972735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:48.972735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:48.972735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:48.972735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:07:48.972735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:07:48.972735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:07:48.972735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:07:48.973837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:48.973837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:48.973837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:07:48.973837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:07:48.973837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:07:48.973837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:07:48.973837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:48.973837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:48.973837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:48.973837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:48.973837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:48.973837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:48.973837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:48.973837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:48.973837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:48.973837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:48.973837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:48.973837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:48.975444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:48.975444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:48.975444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:07:48.975444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:07:48.975444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:07:48.975444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:07:48.975444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:07:48.975444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:07:48.975444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 00:07:48.975444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:48.975444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 244/508] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:48.975444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763272e-1b72-4416-8997-ff7544a11c9d None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:49.021477 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:49.047521 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0425d95-ee6f-465d-9d95-9b1cfc048fbe None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:49.078341 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763272e-1b72-4416-8997-ff7544a11c9d None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:49.081547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f3206b-7c9d-467e-9953-50b115cd01e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:49.082165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3206b-7c9d-467e-9953-50b115cd01e9 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:49.082438 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3206b-7c9d-467e-9953-50b115cd01e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:49.083008 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3206b-7c9d-467e-9953-50b115cd01e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:49.093422 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f3206b-7c9d-467e-9953-50b115cd01e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:49.094871 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f3206b-7c9d-467e-9953-50b115cd01e9 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:49.109048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-36f3206b-7c9d-467e-9953-50b115cd01e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:49.109048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:49.109048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:49.109048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:49.109048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:49.109048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:49.109048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:49.109048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:49.109048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:49.109048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:49.109048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:49.109048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:49.109048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:07:49.109048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:07:49.109048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:07:49.109048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:07:49.109966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:49.109966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:49.109966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:49.109966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:49.109966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:49.109966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:49.109966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:49.109966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:49.109966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:49.109966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:49.109966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:49.109966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:49.109966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:49.109966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:49.109966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:49.109966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:49.109966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:07:49.110799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:07:49.110799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:49.110799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:49.110799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 265/509] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:49.111019 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f3206b-7c9d-467e-9953-50b115cd01e9 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:49.116400 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33086397-5eb7-47c8-bc45-4c9de02e12b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:49.117011 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33086397-5eb7-47c8-bc45-4c9de02e12b5 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:49.117290 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33086397-5eb7-47c8-bc45-4c9de02e12b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:49.117559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33086397-5eb7-47c8-bc45-4c9de02e12b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:49.118222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33086397-5eb7-47c8-bc45-4c9de02e12b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:49.118523 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33086397-5eb7-47c8-bc45-4c9de02e12b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:49.119930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33086397-5eb7-47c8-bc45-4c9de02e12b5 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:49.128894 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 245/510] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 31 00:07:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:49.129689 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33086397-5eb7-47c8-bc45-4c9de02e12b5 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:49.149911 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:49.214792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f3206b-7c9d-467e-9953-50b115cd01e9 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:49.231623 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33086397-5eb7-47c8-bc45-4c9de02e12b5 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:49.234260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:49.251819 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1def2-1d26-48ec-9a39-942f1e9f0092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:49.252423 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1def2-1d26-48ec-9a39-942f1e9f0092 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:49.252697 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1def2-1d26-48ec-9a39-942f1e9f0092 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:49.252962 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1def2-1d26-48ec-9a39-942f1e9f0092 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:49.267459 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df038a5-d673-49f7-9c24-2bdb16b85bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:49.267983 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df038a5-d673-49f7-9c24-2bdb16b85bcf None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:49.268265 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df038a5-d673-49f7-9c24-2bdb16b85bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:49.268542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df038a5-d673-49f7-9c24-2bdb16b85bcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:49.321590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e1def2-1d26-48ec-9a39-942f1e9f0092 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:49.323086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e1def2-1d26-48ec-9a39-942f1e9f0092 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:49.342855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df038a5-d673-49f7-9c24-2bdb16b85bcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:49.344332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df038a5-d673-49f7-9c24-2bdb16b85bcf None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:49.357284 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:49.421357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 266/511] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:49.422453 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1def2-1d26-48ec-9a39-942f1e9f0092 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:49.448612 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 246/512] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:49.449454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df038a5-d673-49f7-9c24-2bdb16b85bcf None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:49.453106 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:49.530635 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1def2-1d26-48ec-9a39-942f1e9f0092 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:49.548928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df038a5-d673-49f7-9c24-2bdb16b85bcf None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:49.564468 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b9b6a-bcdf-44f3-bb5d-d8c4d5b6b4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:49.565161 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b9b6a-bcdf-44f3-bb5d-d8c4d5b6b4f1 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:49.565534 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b9b6a-bcdf-44f3-bb5d-d8c4d5b6b4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:49.565866 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b9b6a-bcdf-44f3-bb5d-d8c4d5b6b4f1 None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:49.566680 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37b9b6a-bcdf-44f3-bb5d-d8c4d5b6b4f1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:49.567079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37b9b6a-bcdf-44f3-bb5d-d8c4d5b6b4f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:49.568586 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37b9b6a-bcdf-44f3-bb5d-d8c4d5b6b4f1 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:49.572058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:49.584909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830de298-20c2-4fad-aa80-be5526569bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:49.585508 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830de298-20c2-4fad-aa80-be5526569bcb None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:49.585787 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830de298-20c2-4fad-aa80-be5526569bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:49.586058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830de298-20c2-4fad-aa80-be5526569bcb None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:49.586699 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830de298-20c2-4fad-aa80-be5526569bcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:49.587003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830de298-20c2-4fad-aa80-be5526569bcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:49.593455 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 267/513] 10.222.0.67 () {56 vars in 1137 bytes} [Wed Aug 31 00:07:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:49.593920 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830de298-20c2-4fad-aa80-be5526569bcb None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:49.594008 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b9b6a-bcdf-44f3-bb5d-d8c4d5b6b4f1 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:49.604291 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 247/514] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:49.605102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830de298-20c2-4fad-aa80-be5526569bcb None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:49.668994 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:49.693627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b9b6a-bcdf-44f3-bb5d-d8c4d5b6b4f1 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:49.707418 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830de298-20c2-4fad-aa80-be5526569bcb None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:49.729274 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9da968-e447-451d-af7c-4ece5520d204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:49.729899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9da968-e447-451d-af7c-4ece5520d204 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:49.730172 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9da968-e447-451d-af7c-4ece5520d204 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:49.730441 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9da968-e447-451d-af7c-4ece5520d204 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:49.740528 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9da968-e447-451d-af7c-4ece5520d204 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:49.741924 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb6a14-3a2d-4d90-99ce-66902f0c16e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:49.742048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9da968-e447-451d-af7c-4ece5520d204 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:49.742546 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb6a14-3a2d-4d90-99ce-66902f0c16e7 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:49.742833 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb6a14-3a2d-4d90-99ce-66902f0c16e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:49.743139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb6a14-3a2d-4d90-99ce-66902f0c16e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:49.753615 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeb6a14-3a2d-4d90-99ce-66902f0c16e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:49.755083 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeb6a14-3a2d-4d90-99ce-66902f0c16e7 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:49.757529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-ba9da968-e447-451d-af7c-4ece5520d204 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:49.757529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:49.757529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:49.757529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:49.757529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:49.757529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:49.757529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:49.757529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:49.757529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:49.757529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:49.757529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:49.757529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:49.757529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:07:49.757529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:07:49.757529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:07:49.757529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:07:49.758644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:49.758644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:49.758644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:49.758644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:49.758644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:49.758644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:49.758644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:49.758644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:49.758644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:49.758644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:49.758644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:49.758644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:49.758644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:49.758644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:49.758644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:49.758644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:49.758644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:07:49.759736 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:07:49.759736 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:49.759736 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:49.759736 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 268/515] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:49.759736 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9da968-e447-451d-af7c-4ece5520d204 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:49.772250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-4eeb6a14-3a2d-4d90-99ce-66902f0c16e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:49.772250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:49.772250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:49.772250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:49.772250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:49.772250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:49.772250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:49.772250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:49.772250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:49.772250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:49.772250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:49.772250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:49.772250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:07:49.772250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:07:49.772250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:07:49.772250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:07:49.773293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:49.773293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:49.773293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:49.773293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:49.773293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:49.773293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:49.773293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:49.773293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:49.773293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:49.773293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:49.773293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:49.773293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:49.773293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:49.773293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:49.773293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:49.773293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:49.773293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:07:49.774260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:07:49.774260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:49.774260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:49.774260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 248/516] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:49.774501 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb6a14-3a2d-4d90-99ce-66902f0c16e7 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:49.809814 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:49.865072 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9da968-e447-451d-af7c-4ece5520d204 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:49.877238 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb6a14-3a2d-4d90-99ce-66902f0c16e7 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:49.893013 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:49.899940 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd4025-e4b4-46f4-9cc7-040c05e6254b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:49.900569 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd4025-e4b4-46f4-9cc7-040c05e6254b None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:49.900848 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd4025-e4b4-46f4-9cc7-040c05e6254b None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:49.901120 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd4025-e4b4-46f4-9cc7-040c05e6254b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:49.911504 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b580e2c5-beef-477f-a79d-d3d61cdecae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:49.911849 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbd4025-e4b4-46f4-9cc7-040c05e6254b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:49.912104 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580e2c5-beef-477f-a79d-d3d61cdecae8 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:49.912381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580e2c5-beef-477f-a79d-d3d61cdecae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:49.912649 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580e2c5-beef-477f-a79d-d3d61cdecae8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:49.913306 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbd4025-e4b4-46f4-9cc7-040c05e6254b None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:49.928000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-2bbd4025-e4b4-46f4-9cc7-040c05e6254b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:49.928000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:49.928000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:49.928000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:49.928000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:49.928000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:49.928000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:49.928000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:49.928000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:49.928000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:49.928000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:49.928000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:49.928000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:07:49.928000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:07:49.928000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:07:49.928000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:07:49.928938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:49.928938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:49.928938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:49.928938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:49.928938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:49.928938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:49.928938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:49.928938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:49.928938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:49.928938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:49.928938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:49.928938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:49.928938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:49.928938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:49.928938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:49.928938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:49.928938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:07:49.929813 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:07:49.929813 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:49.929813 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:49.929813 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 269/517] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:49.930023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd4025-e4b4-46f4-9cc7-040c05e6254b None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:49.984631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b580e2c5-beef-477f-a79d-d3d61cdecae8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:49.986086 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b580e2c5-beef-477f-a79d-d3d61cdecae8 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:50.027361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:50.032214 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd4025-e4b4-46f4-9cc7-040c05e6254b None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:50.066851 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75812b3-494a-446c-82c9-32e01ec7473a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:50.067405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75812b3-494a-446c-82c9-32e01ec7473a None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:50.067690 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75812b3-494a-446c-82c9-32e01ec7473a None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:50.067952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75812b3-494a-446c-82c9-32e01ec7473a None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:50.068589 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75812b3-494a-446c-82c9-32e01ec7473a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:50.070009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75812b3-494a-446c-82c9-32e01ec7473a None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:50.087735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 249/518] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:50.088574 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b580e2c5-beef-477f-a79d-d3d61cdecae8 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:50.095239 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 270/519] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 31 00:07:49 2022] POST /identity/v3/users => generated 247 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:07:50.095935 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75812b3-494a-446c-82c9-32e01ec7473a None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:50.106583 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:50.217428 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b580e2c5-beef-477f-a79d-d3d61cdecae8 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:50.236171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75812b3-494a-446c-82c9-32e01ec7473a None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:50.258092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb483a-9ff7-4801-b378-d95cd8eb6690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:50.258698 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb483a-9ff7-4801-b378-d95cd8eb6690 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:50.258972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb483a-9ff7-4801-b378-d95cd8eb6690 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:50.259333 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb483a-9ff7-4801-b378-d95cd8eb6690 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:50.260141 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb483a-9ff7-4801-b378-d95cd8eb6690 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:50.260250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb483a-9ff7-4801-b378-d95cd8eb6690 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:50.261617 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb483a-9ff7-4801-b378-d95cd8eb6690 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:50.274021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 250/520] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:50.274782 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb483a-9ff7-4801-b378-d95cd8eb6690 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:50.275834 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62198c-383e-4f5c-a282-f3b8015db4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:50.276459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62198c-383e-4f5c-a282-f3b8015db4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:50.276743 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62198c-383e-4f5c-a282-f3b8015db4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:50.277020 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62198c-383e-4f5c-a282-f3b8015db4d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:50.277981 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62198c-383e-4f5c-a282-f3b8015db4d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:50.279142 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62198c-383e-4f5c-a282-f3b8015db4d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:50.281250 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62198c-383e-4f5c-a282-f3b8015db4d2 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:50.300992 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 271/521] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:50.301859 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62198c-383e-4f5c-a282-f3b8015db4d2 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:50.323358 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:50.367673 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:50.394484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb483a-9ff7-4801-b378-d95cd8eb6690 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:50.423860 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62198c-383e-4f5c-a282-f3b8015db4d2 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:50.428895 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b0fd9-4399-4f47-a47d-d5370c072367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:50.429509 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b0fd9-4399-4f47-a47d-d5370c072367 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:50.429789 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b0fd9-4399-4f47-a47d-d5370c072367 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:50.430061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b0fd9-4399-4f47-a47d-d5370c072367 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:50.430704 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b0fd9-4399-4f47-a47d-d5370c072367 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:50.431008 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b0fd9-4399-4f47-a47d-d5370c072367 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:50.432446 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b0fd9-4399-4f47-a47d-d5370c072367 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:50.449092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 251/522] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:50.449092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b0fd9-4399-4f47-a47d-d5370c072367 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:50.470069 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70053d64-aee3-47a0-a2c4-2f78c4b8fcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:50.470958 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70053d64-aee3-47a0-a2c4-2f78c4b8fcaf None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:50.471407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70053d64-aee3-47a0-a2c4-2f78c4b8fcaf None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:50.471847 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70053d64-aee3-47a0-a2c4-2f78c4b8fcaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:50.482687 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70053d64-aee3-47a0-a2c4-2f78c4b8fcaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:50.484383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70053d64-aee3-47a0-a2c4-2f78c4b8fcaf None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:50.500967 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-70053d64-aee3-47a0-a2c4-2f78c4b8fcaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:50.500967 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:50.500967 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:50.500967 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:50.500967 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:50.500967 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:50.500967 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:50.500967 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:50.500967 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:50.500967 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:50.500967 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:50.500967 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:50.500967 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:07:50.500967 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:07:50.500967 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:07:50.500967 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:07:50.501913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:50.501913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:50.501913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:50.501913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:50.501913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:50.501913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:50.501913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:50.501913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:50.501913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:50.501913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:50.501913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:50.501913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:50.501913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:50.501913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:50.501913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:50.501913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:50.501913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:07:50.502744 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:07:50.502744 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:50.502744 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:50.502744 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 272/523] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:50.502954 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70053d64-aee3-47a0-a2c4-2f78c4b8fcaf None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:50.560318 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b0fd9-4399-4f47-a47d-d5370c072367 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:50.573872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:50.600152 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7e000-f888-4aeb-aed3-5fd66c8149cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:50.600790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7e000-f888-4aeb-aed3-5fd66c8149cc None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:50.601066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7e000-f888-4aeb-aed3-5fd66c8149cc None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:50.601338 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7e000-f888-4aeb-aed3-5fd66c8149cc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:50.604510 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:50.610365 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7e000-f888-4aeb-aed3-5fd66c8149cc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:50.612297 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7e000-f888-4aeb-aed3-5fd66c8149cc None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:50.615974 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70053d64-aee3-47a0-a2c4-2f78c4b8fcaf None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:50.626442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-f9f7e000-f888-4aeb-aed3-5fd66c8149cc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 00:07:50.626442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:50.626442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:50.626442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:50.626442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:50.626442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:50.626442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:50.626442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:50.626442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:50.626442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:50.626442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:50.626442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:50.626442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:07:50.626442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:07:50.626442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:07:50.626442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:07:50.627506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:50.627506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:50.627506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:07:50.627506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:07:50.627506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:07:50.627506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:07:50.627506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:50.627506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:50.627506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:50.627506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:50.627506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:50.627506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:50.627506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:50.627506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:50.627506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:50.627506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:50.627506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:50.627506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:50.628408 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:50.628408 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:50.628408 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:07:50.628408 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:07:50.628408 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:07:50.628408 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:07:50.628408 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:07:50.628408 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:07:50.628408 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 00:07:50.628408 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:50.633955 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 252/524] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 31 00:07:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:50.633955 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7e000-f888-4aeb-aed3-5fd66c8149cc None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:50.651606 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eebaad4-a860-4112-853a-40ca76c1bd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:50.652044 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebaad4-a860-4112-853a-40ca76c1bd15 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:50.652325 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebaad4-a860-4112-853a-40ca76c1bd15 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:50.652595 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebaad4-a860-4112-853a-40ca76c1bd15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:50.661990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eebaad4-a860-4112-853a-40ca76c1bd15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:50.671695 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eebaad4-a860-4112-853a-40ca76c1bd15 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:50.689501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-7eebaad4-a860-4112-853a-40ca76c1bd15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:07:50.689501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:50.689501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:50.689501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:50.689501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:50.689501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:50.689501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:50.689501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:50.689501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:50.689501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:50.689501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:50.689501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:50.689501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:07:50.689501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:07:50.689501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:07:50.690417 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:07:50.690417 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:50.690417 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:50.690417 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:50.690417 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:50.690417 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:50.690417 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:50.690417 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:50.690417 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:50.690417 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:50.690417 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:50.690417 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:50.690417 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:50.690417 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:50.690417 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:50.690417 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:50.690417 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:50.692049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:07:50.692049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:07:50.692049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:07:50.692049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:07:50.692049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:07:50.692049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:07:50.692049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:07:50.692049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:50.692049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 273/525] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:07:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:50.692049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eebaad4-a860-4112-853a-40ca76c1bd15 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:50.759326 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7e000-f888-4aeb-aed3-5fd66c8149cc None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:50.796560 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a3513-2d5c-4a5e-a2bb-79defcf1cadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:50.797193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a3513-2d5c-4a5e-a2bb-79defcf1cadc None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:50.797469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a3513-2d5c-4a5e-a2bb-79defcf1cadc None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:50.797740 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a3513-2d5c-4a5e-a2bb-79defcf1cadc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:50.801159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eebaad4-a860-4112-853a-40ca76c1bd15 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:50.805263 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a3513-2d5c-4a5e-a2bb-79defcf1cadc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:50.806717 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a3513-2d5c-4a5e-a2bb-79defcf1cadc None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:50.820254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-444a3513-2d5c-4a5e-a2bb-79defcf1cadc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:07:50.820254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:50.820254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:50.820254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:50.820254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:50.820254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:50.820254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:50.820254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:50.820254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:50.820254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:50.820254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:50.820254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:50.820254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:07:50.820254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:07:50.820254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:07:50.821147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:07:50.821147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:50.821147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:50.821147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:50.821147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:50.821147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:50.821147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:50.821147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:50.821147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:50.821147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:50.821147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:50.821147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:50.821147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:50.821147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:50.821147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:50.821147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:50.821147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:50.821979 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:07:50.821979 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:07:50.821979 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:07:50.821979 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:07:50.821979 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:07:50.821979 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:07:50.821979 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:07:50.821979 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:50.826939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 253/526] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:07:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:50.827626 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a3513-2d5c-4a5e-a2bb-79defcf1cadc None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:50.830606 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:50.840602 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f2770-6b22-4d42-9876-1ffb38cd9522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:50.841264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f2770-6b22-4d42-9876-1ffb38cd9522 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:50.841597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f2770-6b22-4d42-9876-1ffb38cd9522 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:50.841868 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f2770-6b22-4d42-9876-1ffb38cd9522 None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:50.842432 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f2770-6b22-4d42-9876-1ffb38cd9522 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:50.842733 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f2770-6b22-4d42-9876-1ffb38cd9522 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:50.844432 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f2770-6b22-4d42-9876-1ffb38cd9522 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:50.857077 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 274/527] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:50.857873 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f2770-6b22-4d42-9876-1ffb38cd9522 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:50.888615 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:50.940387 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a3513-2d5c-4a5e-a2bb-79defcf1cadc None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:50.971222 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f2770-6b22-4d42-9876-1ffb38cd9522 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:50.988412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d022c-e5e2-4018-b811-518ad932f04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:50.989724 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d022c-e5e2-4018-b811-518ad932f04f None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:50.990005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d022c-e5e2-4018-b811-518ad932f04f None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:50.990280 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d022c-e5e2-4018-b811-518ad932f04f None admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:50.991017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d022c-e5e2-4018-b811-518ad932f04f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:50.991358 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d022c-e5e2-4018-b811-518ad932f04f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:50.992856 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d022c-e5e2-4018-b811-518ad932f04f None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:51.012288 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 254/528] 10.222.0.67 () {56 vars in 1139 bytes} [Wed Aug 31 00:07:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:51.017467 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d022c-e5e2-4018-b811-518ad932f04f None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:51.021810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1873af64-ecc5-4c1e-8371-ef5197f037d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:51.022423 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873af64-ecc5-4c1e-8371-ef5197f037d6 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:51.022717 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873af64-ecc5-4c1e-8371-ef5197f037d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:51.022982 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873af64-ecc5-4c1e-8371-ef5197f037d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:51.023712 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1873af64-ecc5-4c1e-8371-ef5197f037d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:51.024017 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1873af64-ecc5-4c1e-8371-ef5197f037d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:51.025444 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1873af64-ecc5-4c1e-8371-ef5197f037d6 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:51.044688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 275/529] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:07:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:51.045522 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1873af64-ecc5-4c1e-8371-ef5197f037d6 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:51.097135 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:51.133185 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d022c-e5e2-4018-b811-518ad932f04f None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:51.151628 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:51.159409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1873af64-ecc5-4c1e-8371-ef5197f037d6 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:51.174361 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e8bf55-2a8a-440c-84d4-20fd5835752e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:51.175052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8bf55-2a8a-440c-84d4-20fd5835752e None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:51.175346 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8bf55-2a8a-440c-84d4-20fd5835752e None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:51.175617 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8bf55-2a8a-440c-84d4-20fd5835752e None admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:51.176306 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e8bf55-2a8a-440c-84d4-20fd5835752e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:51.176608 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e8bf55-2a8a-440c-84d4-20fd5835752e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:51.178242 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e8bf55-2a8a-440c-84d4-20fd5835752e None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:51.195358 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d709f-c6d4-4bcc-b24e-f596a94ac94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:51.195989 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d709f-c6d4-4bcc-b24e-f596a94ac94c None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:51.201557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d709f-c6d4-4bcc-b24e-f596a94ac94c None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:51.201557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d709f-c6d4-4bcc-b24e-f596a94ac94c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:51.201799 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 255/530] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:07:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:51.201799 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e8bf55-2a8a-440c-84d4-20fd5835752e None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:51.208087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32d709f-c6d4-4bcc-b24e-f596a94ac94c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:51.209546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32d709f-c6d4-4bcc-b24e-f596a94ac94c None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:51.221805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-a32d709f-c6d4-4bcc-b24e-f596a94ac94c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:07:51.221805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:51.221805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:51.221805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:51.221805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:51.221805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:51.221805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:51.221805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:51.221805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:51.221805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:51.221805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:51.221805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:51.221805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:07:51.221805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:07:51.221805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:07:51.222718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:07:51.222718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:51.222718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:51.222718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:51.222718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:51.222718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:51.222718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:51.222718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:51.222718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:51.222718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:51.222718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:51.222718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:51.222718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:51.222718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:51.222718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:51.222718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:51.222718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:51.223960 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:07:51.223960 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:07:51.223960 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:07:51.223960 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:07:51.223960 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:07:51.223960 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:07:51.223960 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:07:51.223960 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:51.223960 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 276/531] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:07:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:51.223960 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d709f-c6d4-4bcc-b24e-f596a94ac94c None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:51.322131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e8bf55-2a8a-440c-84d4-20fd5835752e None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:51.337205 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d709f-c6d4-4bcc-b24e-f596a94ac94c None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:51.347957 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:51.358664 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5687c6-7c80-4552-b39b-fcffbf1b7ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:51.359303 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5687c6-7c80-4552-b39b-fcffbf1b7ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:51.359593 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5687c6-7c80-4552-b39b-fcffbf1b7ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:51.359882 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5687c6-7c80-4552-b39b-fcffbf1b7ad9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:51.384075 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2cb8b-c803-4573-a15c-69695f360a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:51.385367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2cb8b-c803-4573-a15c-69695f360a2a None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:51.385862 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2cb8b-c803-4573-a15c-69695f360a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:51.386069 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2cb8b-c803-4573-a15c-69695f360a2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:51.386791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a2cb8b-c803-4573-a15c-69695f360a2a None admin] RBAC: Adding query filter params (role.id=4680aa64da80470b87075090fca2354b, scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb, user.id=3aa53ae5a30146f69d7d6eefe4290dba) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:51.387129 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a2cb8b-c803-4573-a15c-69695f360a2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:51.388745 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a2cb8b-c803-4573-a15c-69695f360a2a None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:51.403126 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 277/532] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:07:51 2022] GET /identity/v3/role_assignments?user.id=3aa53ae5a30146f69d7d6eefe4290dba&scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb&role.id=4680aa64da80470b87075090fca2354b => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:51.403126 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2cb8b-c803-4573-a15c-69695f360a2a None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:51.408542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:51.444994 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5687c6-7c80-4552-b39b-fcffbf1b7ad9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:51.446593 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5687c6-7c80-4552-b39b-fcffbf1b7ad9 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:51.543156 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2cb8b-c803-4573-a15c-69695f360a2a None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:51.563750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 256/533] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:51.564600 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5687c6-7c80-4552-b39b-fcffbf1b7ad9 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:51.591550 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808dc7e-89cd-4e3f-9ae7-7fefab50222e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:51.592308 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808dc7e-89cd-4e3f-9ae7-7fefab50222e None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:51.592684 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808dc7e-89cd-4e3f-9ae7-7fefab50222e None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:51.593054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808dc7e-89cd-4e3f-9ae7-7fefab50222e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:51.593882 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808dc7e-89cd-4e3f-9ae7-7fefab50222e None admin] RBAC: Adding query filter params (role.id=4680aa64da80470b87075090fca2354b, scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb, user.id=55714a21c1cb4f2b949603230243ed02) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:51.594287 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808dc7e-89cd-4e3f-9ae7-7fefab50222e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:51.595966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808dc7e-89cd-4e3f-9ae7-7fefab50222e None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:51.610307 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 278/534] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:07:51 2022] GET /identity/v3/role_assignments?user.id=55714a21c1cb4f2b949603230243ed02&scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb&role.id=4680aa64da80470b87075090fca2354b => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:51.612317 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808dc7e-89cd-4e3f-9ae7-7fefab50222e None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:51.656172 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:51.739754 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:51.757140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808dc7e-89cd-4e3f-9ae7-7fefab50222e None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:51.801342 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5687c6-7c80-4552-b39b-fcffbf1b7ad9 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:51.823092 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851d678-b3a8-4071-b020-1583e1ffe44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:51.823736 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851d678-b3a8-4071-b020-1583e1ffe44b None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:51.824015 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851d678-b3a8-4071-b020-1583e1ffe44b None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:51.824288 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851d678-b3a8-4071-b020-1583e1ffe44b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:51.846618 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851d678-b3a8-4071-b020-1583e1ffe44b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:51.848333 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851d678-b3a8-4071-b020-1583e1ffe44b None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:51.853443 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42389c-b0d4-47d6-a2fa-83d8599b400c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:51.854061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42389c-b0d4-47d6-a2fa-83d8599b400c None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:51.854339 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42389c-b0d4-47d6-a2fa-83d8599b400c None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:51.854609 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42389c-b0d4-47d6-a2fa-83d8599b400c None admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:51.855327 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c42389c-b0d4-47d6-a2fa-83d8599b400c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:51.855637 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c42389c-b0d4-47d6-a2fa-83d8599b400c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:51.857195 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c42389c-b0d4-47d6-a2fa-83d8599b400c None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:51.876720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-e851d678-b3a8-4071-b020-1583e1ffe44b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:51.876720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:51.876720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:51.876720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:51.876720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:51.876720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:51.876720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:51.876720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:51.876720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:51.876720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:51.876720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:51.876720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:51.876720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:07:51.876720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:07:51.876720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:07:51.876720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:07:51.877865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:51.877865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:51.877865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:51.877865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:51.877865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:51.877865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:51.877865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:51.877865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:51.877865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:51.877865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:51.877865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:51.877865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:51.877865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:51.877865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:51.877865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:51.877865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:51.877865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:07:51.878897 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:07:51.878897 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:51.878897 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:51.881502 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 279/535] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:51.890948 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 257/536] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:07:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:51.891410 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851d678-b3a8-4071-b020-1583e1ffe44b None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:51.893283 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42389c-b0d4-47d6-a2fa-83d8599b400c None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:52.051503 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42389c-b0d4-47d6-a2fa-83d8599b400c None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:52.062694 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:52.092927 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851d678-b3a8-4071-b020-1583e1ffe44b None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:52.120275 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77018e25-4d80-4358-a9b9-27d2ef56f865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:52.120893 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77018e25-4d80-4358-a9b9-27d2ef56f865 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:52.121170 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77018e25-4d80-4358-a9b9-27d2ef56f865 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:52.121441 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77018e25-4d80-4358-a9b9-27d2ef56f865 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:52.122255 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77018e25-4d80-4358-a9b9-27d2ef56f865 None admin] RBAC: Adding query filter params (role.id=4680aa64da80470b87075090fca2354b, scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb, user.id=7e42dfb7d22e4435889cb91327db5477) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:52.122535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77018e25-4d80-4358-a9b9-27d2ef56f865 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:52.123995 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77018e25-4d80-4358-a9b9-27d2ef56f865 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:52.131094 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 258/537] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:07:51 2022] GET /identity/v3/role_assignments?user.id=7e42dfb7d22e4435889cb91327db5477&scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb&role.id=4680aa64da80470b87075090fca2354b => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:52.131566 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:52.159500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df0e168-dea9-48ba-aac2-aa9876b05717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:52.160136 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0e168-dea9-48ba-aac2-aa9876b05717 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:52.160412 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0e168-dea9-48ba-aac2-aa9876b05717 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:52.160681 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0e168-dea9-48ba-aac2-aa9876b05717 None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:52.161351 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df0e168-dea9-48ba-aac2-aa9876b05717 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:52.161654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df0e168-dea9-48ba-aac2-aa9876b05717 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:52.163122 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df0e168-dea9-48ba-aac2-aa9876b05717 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:52.179297 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 280/538] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:52.184581 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77018e25-4d80-4358-a9b9-27d2ef56f865 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:52.259764 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecaacc2-f07b-487f-b39a-f88d0981851f None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:52.260049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecaacc2-f07b-487f-b39a-f88d0981851f None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:52.260328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecaacc2-f07b-487f-b39a-f88d0981851f None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:52.261769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 281/539] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:07:52.267664 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d885bb-6448-4ffe-bf92-150ed72fef4b None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:52.268035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d885bb-6448-4ffe-bf92-150ed72fef4b None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:52.268390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d885bb-6448-4ffe-bf92-150ed72fef4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:52.312254 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-57d885bb-6448-4ffe-bf92-150ed72fef4b None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:52.313478 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-57d885bb-6448-4ffe-bf92-150ed72fef4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:52.332942 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77018e25-4d80-4358-a9b9-27d2ef56f865 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:52.375706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513862a3-58f5-4b44-b7b6-58e3ecb6487d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:52.376640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513862a3-58f5-4b44-b7b6-58e3ecb6487d None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:52.377121 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513862a3-58f5-4b44-b7b6-58e3ecb6487d None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:52.377585 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513862a3-58f5-4b44-b7b6-58e3ecb6487d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:52.385028 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513862a3-58f5-4b44-b7b6-58e3ecb6487d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:52.386724 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513862a3-58f5-4b44-b7b6-58e3ecb6487d None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:52.407063 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 282/540] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:07:52.407251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-513862a3-58f5-4b44-b7b6-58e3ecb6487d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:07:52.407251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:52.407251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:52.407251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:52.407251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:52.407251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:52.407251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:52.407251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:52.407251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:52.407251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:52.407251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:52.407251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:52.407251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:07:52.407251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:07:52.407251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:07:52.408040 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:07:52.408040 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:52.408040 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:52.408040 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:52.408040 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:52.408040 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:52.408040 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:52.408040 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:52.408040 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:52.408040 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:52.408040 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:52.408040 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:52.408040 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:52.408040 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:52.408040 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:52.408040 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:52.408040 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:52.408884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:07:52.408884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:07:52.408884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:07:52.408884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:07:52.408884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:07:52.408884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:07:52.408884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:07:52.408884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:52.408884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 259/541] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:07:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:52.412487 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d885bb-6448-4ffe-bf92-150ed72fef4b None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:52.504113 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:52.529679 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d885bb-6448-4ffe-bf92-150ed72fef4b None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:52.569728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861c9f9-504e-4aeb-a3bd-70e59ff7d47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:52.570340 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861c9f9-504e-4aeb-a3bd-70e59ff7d47e None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:52.570618 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861c9f9-504e-4aeb-a3bd-70e59ff7d47e None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:52.570889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861c9f9-504e-4aeb-a3bd-70e59ff7d47e None admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:52.571614 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861c9f9-504e-4aeb-a3bd-70e59ff7d47e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:52.571916 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861c9f9-504e-4aeb-a3bd-70e59ff7d47e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:52.576432 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861c9f9-504e-4aeb-a3bd-70e59ff7d47e None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:52.588875 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 283/542] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:07:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:52.595478 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513862a3-58f5-4b44-b7b6-58e3ecb6487d None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:52.671384 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:52.714347 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513862a3-58f5-4b44-b7b6-58e3ecb6487d None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:52.749299 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ccb43-04ee-432b-87ae-287ec33ba2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:52.749986 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ccb43-04ee-432b-87ae-287ec33ba2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:52.750530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ccb43-04ee-432b-87ae-287ec33ba2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:52.750799 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ccb43-04ee-432b-87ae-287ec33ba2b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:52.751598 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ccb43-04ee-432b-87ae-287ec33ba2b0 None admin] RBAC: Adding query filter params (role.id=4680aa64da80470b87075090fca2354b, scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb, user.id=05c0d9554cf94c41a2cd709cab378e53) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:52.751909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ccb43-04ee-432b-87ae-287ec33ba2b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:52.753345 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ccb43-04ee-432b-87ae-287ec33ba2b0 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:52.759453 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 260/543] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:07:52 2022] GET /identity/v3/role_assignments?user.id=05c0d9554cf94c41a2cd709cab378e53&scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb&role.id=4680aa64da80470b87075090fca2354b => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:52.837177 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861c9f9-504e-4aeb-a3bd-70e59ff7d47e None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:52.961508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861c9f9-504e-4aeb-a3bd-70e59ff7d47e None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:53.002760 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df774f3b-8d54-403a-ad25-f913a24a65b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:53.003430 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df774f3b-8d54-403a-ad25-f913a24a65b4 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:53.003712 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df774f3b-8d54-403a-ad25-f913a24a65b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:53.003983 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df774f3b-8d54-403a-ad25-f913a24a65b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:53.006744 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:53.017193 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df774f3b-8d54-403a-ad25-f913a24a65b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:53.022469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-df774f3b-8d54-403a-ad25-f913a24a65b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:07:53.022469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:53.022469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:53.022469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:53.022469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:53.022469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:53.022469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:53.022469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:53.022469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:53.022469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:53.022469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:53.022469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:53.022469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:53.022469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:53.022469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:53.022469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:53.023362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:53.023362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:53.023362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:53.023362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:53.023362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:53.023362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:53.023362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:53.023362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:53.023362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:53.023362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:53.023362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:53.023362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:53.023362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:53.023362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:53.023362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:53.023362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:53.024159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:53.024159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:53.024159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:53.024159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:53.024159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:53.024159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:53.024159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:53.024159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:53.024159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:53.024159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:53.024159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:07:53.024159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:53.024159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 284/544] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 31 00:07:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:53.029869 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ccb43-04ee-432b-87ae-287ec33ba2b0 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:53.162342 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ccb43-04ee-432b-87ae-287ec33ba2b0 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:53.186887 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:53.200573 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fbd56-2490-4ff0-9069-2b12464fb7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:53.201189 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fbd56-2490-4ff0-9069-2b12464fb7d1 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:53.201472 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fbd56-2490-4ff0-9069-2b12464fb7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:53.201744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fbd56-2490-4ff0-9069-2b12464fb7d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:53.202408 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fbd56-2490-4ff0-9069-2b12464fb7d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:53.202711 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fbd56-2490-4ff0-9069-2b12464fb7d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:53.204131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fbd56-2490-4ff0-9069-2b12464fb7d1 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:53.213022 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 261/545] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 31 00:07:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:53.219210 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df774f3b-8d54-403a-ad25-f913a24a65b4 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:53.349853 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df774f3b-8d54-403a-ad25-f913a24a65b4 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:53.388715 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5cc8e-b822-44ea-9dbe-6f01d8035884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:53.389357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5cc8e-b822-44ea-9dbe-6f01d8035884 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:53.389641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5cc8e-b822-44ea-9dbe-6f01d8035884 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:53.389915 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5cc8e-b822-44ea-9dbe-6f01d8035884 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:53.405994 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e5cc8e-b822-44ea-9dbe-6f01d8035884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:53.407518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e5cc8e-b822-44ea-9dbe-6f01d8035884 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:53.429357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-50e5cc8e-b822-44ea-9dbe-6f01d8035884 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:53.429357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:53.429357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:53.429357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:53.429357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:53.429357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:53.429357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:53.429357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:53.429357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:53.429357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:53.429357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:53.429357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:53.429357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:07:53.429357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:07:53.429357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:07:53.429357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:07:53.430287 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:53.430287 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:53.430287 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:53.430287 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:53.430287 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:53.430287 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:53.430287 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:53.430287 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:53.430287 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:53.430287 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:53.430287 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:53.430287 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:53.430287 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:53.430287 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:53.430287 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:53.430287 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:53.430287 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:07:53.431136 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:07:53.431136 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:53.431136 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:53.431136 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 285/546] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:53.437060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fbd56-2490-4ff0-9069-2b12464fb7d1 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:53.570042 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fbd56-2490-4ff0-9069-2b12464fb7d1 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:53.608777 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b7d4f-2c2c-4d60-82e7-8b1f7731f146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:53.609387 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b7d4f-2c2c-4d60-82e7-8b1f7731f146 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:53.609666 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b7d4f-2c2c-4d60-82e7-8b1f7731f146 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:53.609941 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b7d4f-2c2c-4d60-82e7-8b1f7731f146 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:53.610579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065b7d4f-2c2c-4d60-82e7-8b1f7731f146 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:53.610881 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065b7d4f-2c2c-4d60-82e7-8b1f7731f146 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:53.612283 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065b7d4f-2c2c-4d60-82e7-8b1f7731f146 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:53.625378 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 262/547] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:07:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:53.630832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5cc8e-b822-44ea-9dbe-6f01d8035884 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:53.641524 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f5c68-e47c-479f-bf27-4344ce5606ca None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:53.641812 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f5c68-e47c-479f-bf27-4344ce5606ca None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:53.642649 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f5c68-e47c-479f-bf27-4344ce5606ca None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:53.646182 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 263/548] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:53 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:53.649221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b83d1c-5082-4eb5-9e72-17c160efa5f0 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:53.649501 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b83d1c-5082-4eb5-9e72-17c160efa5f0 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:53.649766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b83d1c-5082-4eb5-9e72-17c160efa5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:53.682956 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-d8b83d1c-5082-4eb5-9e72-17c160efa5f0 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:53.684177 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-d8b83d1c-5082-4eb5-9e72-17c160efa5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:53.723426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:53.766885 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:53.804401 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 264/549] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:53.809811 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5cc8e-b822-44ea-9dbe-6f01d8035884 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:53.854534 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa4605-a4cc-46ec-84e7-28ca4da9bc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:53.855944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa4605-a4cc-46ec-84e7-28ca4da9bc82 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:53.856461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa4605-a4cc-46ec-84e7-28ca4da9bc82 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:53.856907 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa4605-a4cc-46ec-84e7-28ca4da9bc82 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:53.870661 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aa4605-a4cc-46ec-84e7-28ca4da9bc82 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:53.872908 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aa4605-a4cc-46ec-84e7-28ca4da9bc82 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:53.890902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-b8aa4605-a4cc-46ec-84e7-28ca4da9bc82 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 00:07:53.890902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:53.890902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:53.890902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:53.890902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:53.890902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:53.890902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:53.890902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:53.890902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:53.890902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:53.890902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:53.890902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:53.890902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:07:53.890902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:07:53.890902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:07:53.890902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:07:53.892311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:53.892311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:53.892311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:07:53.892311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:07:53.892311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:07:53.892311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:07:53.892311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:53.892311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:53.892311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:53.892311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:53.892311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:53.892311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:53.892311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:53.892311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:53.892311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:53.892311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:53.892311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:53.892311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:53.893356 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:53.893356 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:53.893356 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:07:53.893356 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:07:53.893356 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:07:53.893356 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:07:53.893356 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:07:53.893356 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:07:53.893356 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 00:07:53.893356 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:53.899192 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 286/550] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:07:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:53.905667 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b83d1c-5082-4eb5-9e72-17c160efa5f0 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:54.072693 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b83d1c-5082-4eb5-9e72-17c160efa5f0 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:54.117594 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4001ba-bcb7-4ea4-a37e-6d3ecc091214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:54.118135 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4001ba-bcb7-4ea4-a37e-6d3ecc091214 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:54.118414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4001ba-bcb7-4ea4-a37e-6d3ecc091214 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:54.118687 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4001ba-bcb7-4ea4-a37e-6d3ecc091214 None admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:54.119497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4001ba-bcb7-4ea4-a37e-6d3ecc091214 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:54.119810 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4001ba-bcb7-4ea4-a37e-6d3ecc091214 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:54.121342 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4001ba-bcb7-4ea4-a37e-6d3ecc091214 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:54.146205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 265/551] 10.222.0.67 () {56 vars in 1135 bytes} [Wed Aug 31 00:07:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:54.151922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa4605-a4cc-46ec-84e7-28ca4da9bc82 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:54.165422 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4001ba-bcb7-4ea4-a37e-6d3ecc091214 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:54.219456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:54.272109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4001ba-bcb7-4ea4-a37e-6d3ecc091214 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:54.272522 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa4605-a4cc-46ec-84e7-28ca4da9bc82 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:54.282144 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:54.310364 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdc9f1-f4ec-4ae7-8403-cfb3d7e18e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:54.310568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c53ff4-0340-4c85-82c0-7594b805074f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:54.311207 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c53ff4-0340-4c85-82c0-7594b805074f None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:54.311310 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdc9f1-f4ec-4ae7-8403-cfb3d7e18e5d None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:54.311486 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c53ff4-0340-4c85-82c0-7594b805074f None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:54.311601 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdc9f1-f4ec-4ae7-8403-cfb3d7e18e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:54.311749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c53ff4-0340-4c85-82c0-7594b805074f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:54.311836 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdc9f1-f4ec-4ae7-8403-cfb3d7e18e5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:54.322835 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c53ff4-0340-4c85-82c0-7594b805074f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:54.328962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-f9c53ff4-0340-4c85-82c0-7594b805074f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:07:54.328962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:54.328962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:54.328962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:54.328962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:54.328962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:54.328962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:54.328962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:54.328962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:54.328962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:54.328962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:54.328962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:54.328962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:54.328962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:54.328962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:54.328962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:54.330034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:54.330034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:54.330034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:54.330034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:54.330034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:54.330034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:54.330034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:54.330034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:54.330034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:54.330034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:54.330034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:54.330034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:54.330034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:54.330034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:54.330034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:54.330034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:54.331254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:54.331254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:54.331254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:54.331254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:54.331254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:54.331254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:54.331254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:54.331254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:54.331254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:54.331254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:54.331254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:07:54.331254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:54.331254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 266/552] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 31 00:07:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:07:54.332237 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe307e7-4b2e-4be4-a75d-111e8608a520 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:54.332237 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe307e7-4b2e-4be4-a75d-111e8608a520 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:54.332237 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe307e7-4b2e-4be4-a75d-111e8608a520 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:54.337832 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 267/553] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:54.339123 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2ddbf-918d-4776-8675-e44b15f1bbea None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:54.339409 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2ddbf-918d-4776-8675-e44b15f1bbea None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:54.339785 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2ddbf-918d-4776-8675-e44b15f1bbea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:54.361936 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-15b2ddbf-918d-4776-8675-e44b15f1bbea None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:54.363152 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-15b2ddbf-918d-4776-8675-e44b15f1bbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:54.408554 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fdc9f1-f4ec-4ae7-8403-cfb3d7e18e5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:54.409344 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fdc9f1-f4ec-4ae7-8403-cfb3d7e18e5d None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:54.445530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 268/554] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:54.446464 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2ddbf-918d-4776-8675-e44b15f1bbea None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:54.474330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:54.565177 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2ddbf-918d-4776-8675-e44b15f1bbea None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:54.604552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 287/555] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:07:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:07:54.612561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdc9f1-f4ec-4ae7-8403-cfb3d7e18e5d None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:54.612801 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe2642-d5e9-4a96-8e93-1ae394f76f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:54.612801 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe2642-d5e9-4a96-8e93-1ae394f76f03 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:54.613225 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe2642-d5e9-4a96-8e93-1ae394f76f03 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:54.613225 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe2642-d5e9-4a96-8e93-1ae394f76f03 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:54.614753 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fe2642-d5e9-4a96-8e93-1ae394f76f03 None admin] RBAC: Adding query filter params (name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:54.615082 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fe2642-d5e9-4a96-8e93-1ae394f76f03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:54.616528 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fe2642-d5e9-4a96-8e93-1ae394f76f03 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:54.623413 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:54.633596 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 269/556] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 31 00:07:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:54.638172 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe2642-d5e9-4a96-8e93-1ae394f76f03 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:54.781949 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:54.783403 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdc9f1-f4ec-4ae7-8403-cfb3d7e18e5d None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:54.804175 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe2642-d5e9-4a96-8e93-1ae394f76f03 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:54.830809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6040e7af-3bac-4779-bad6-9e189e94baca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:54.831791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6040e7af-3bac-4779-bad6-9e189e94baca None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:54.832075 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6040e7af-3bac-4779-bad6-9e189e94baca None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:54.832345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6040e7af-3bac-4779-bad6-9e189e94baca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:54.842566 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f524c16-2aab-4fca-8ca2-2b26e04b240b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:54.843230 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6040e7af-3bac-4779-bad6-9e189e94baca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:54.843340 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f524c16-2aab-4fca-8ca2-2b26e04b240b None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:54.843529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f524c16-2aab-4fca-8ca2-2b26e04b240b None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:54.843781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f524c16-2aab-4fca-8ca2-2b26e04b240b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:54.844681 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6040e7af-3bac-4779-bad6-9e189e94baca None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:54.854630 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f524c16-2aab-4fca-8ca2-2b26e04b240b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:54.856131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f524c16-2aab-4fca-8ca2-2b26e04b240b None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:54.860369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-6040e7af-3bac-4779-bad6-9e189e94baca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:54.860369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:54.860369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:54.860369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:54.860369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:54.860369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:54.860369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:54.860369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:54.860369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:54.860369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:54.860369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:54.860369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:54.860369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:07:54.860369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:07:54.860369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:07:54.860369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:07:54.861636 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:54.861636 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:54.861636 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:54.861636 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:54.861636 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:54.861636 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:54.861636 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:54.861636 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:54.861636 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:54.861636 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:54.861636 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:54.861636 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:54.861636 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:54.861636 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:54.861636 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:54.861636 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:54.861636 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:07:54.862602 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:07:54.862602 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:54.862602 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:54.862602 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 288/557] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:54.862602 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6040e7af-3bac-4779-bad6-9e189e94baca None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:54.877149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-9f524c16-2aab-4fca-8ca2-2b26e04b240b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:54.877149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:54.877149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:54.877149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:54.877149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:54.877149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:54.877149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:54.877149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:54.877149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:54.877149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:54.877149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:54.877149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:54.877149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:07:54.877149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:07:54.877149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:07:54.877149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:07:54.878580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:54.878580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:54.878580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:54.878580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:54.878580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:54.878580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:54.878580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:54.878580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:54.878580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:54.878580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:54.878580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:54.878580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:54.878580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:54.878580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:54.878580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:54.878580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:54.878580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:07:54.879525 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:07:54.879525 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:54.879525 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:54.879525 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 270/558] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:54.880199 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e20588-8aca-43ab-9ba0-8465d4e7b449 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:54.880199 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e20588-8aca-43ab-9ba0-8465d4e7b449 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:54.880429 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e20588-8aca-43ab-9ba0-8465d4e7b449 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:54.883082 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 271/559] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:54.883082 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e20588-8aca-43ab-9ba0-8465d4e7b449 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:54.936880 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:54.983736 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6040e7af-3bac-4779-bad6-9e189e94baca None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:54.998893 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e20588-8aca-43ab-9ba0-8465d4e7b449 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:55.022117 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e146b2a-13c9-46e0-8596-3d600d8bce67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:55.022741 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e146b2a-13c9-46e0-8596-3d600d8bce67 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:55.023018 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e146b2a-13c9-46e0-8596-3d600d8bce67 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:55.023314 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e146b2a-13c9-46e0-8596-3d600d8bce67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:55.033685 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e146b2a-13c9-46e0-8596-3d600d8bce67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:55.033986 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04a5cc-dbf7-4568-8449-64d6c14e303b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:55.034588 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04a5cc-dbf7-4568-8449-64d6c14e303b None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:55.034863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04a5cc-dbf7-4568-8449-64d6c14e303b None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:55.035157 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04a5cc-dbf7-4568-8449-64d6c14e303b None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:55.035790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04a5cc-dbf7-4568-8449-64d6c14e303b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:55.036087 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04a5cc-dbf7-4568-8449-64d6c14e303b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:55.037439 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04a5cc-dbf7-4568-8449-64d6c14e303b None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:55.038748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-2e146b2a-13c9-46e0-8596-3d600d8bce67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:07:55.038748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:55.038748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:55.038748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:55.038748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:55.038748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:55.038748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:55.038748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:55.038748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:55.038748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:55.038748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:55.038748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:55.038748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:55.038748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:55.038748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:55.038748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:55.039829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:55.039829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:55.039829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:55.039829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:55.039829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:55.039829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:55.039829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:55.039829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:55.039829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:55.039829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:55.039829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:55.039829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:55.039829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:55.039829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:55.039829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:55.039829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:55.040645 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:55.040645 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:55.040645 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:55.040645 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:55.040645 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:55.040645 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:55.040645 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:55.040645 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:55.040645 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:55.040645 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:55.040645 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:07:55.040645 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:55.040645 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 289/560] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 31 00:07:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:55.041326 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:55.041396 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e146b2a-13c9-46e0-8596-3d600d8bce67 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:55.054321 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 272/561] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:55.055574 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0404a-a6ff-4d39-8a60-71d310a3e07d None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:55.055862 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0404a-a6ff-4d39-8a60-71d310a3e07d None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:55.056131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0404a-a6ff-4d39-8a60-71d310a3e07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:55.073564 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-94a0404a-a6ff-4d39-8a60-71d310a3e07d None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:55.074744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-94a0404a-a6ff-4d39-8a60-71d310a3e07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:55.141575 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e146b2a-13c9-46e0-8596-3d600d8bce67 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:55.141951 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 273/562] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:55.143231 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cdd6c-c51b-456a-831e-ec2f849e3eaf None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:55.143523 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cdd6c-c51b-456a-831e-ec2f849e3eaf None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:55.143802 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cdd6c-c51b-456a-831e-ec2f849e3eaf None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:55.145216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 274/563] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:55.145860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5cdd6c-c51b-456a-831e-ec2f849e3eaf None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:55.167390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:55.183822 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86123a-587f-4db8-af1d-5e7038a0257e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:55.184566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86123a-587f-4db8-af1d-5e7038a0257e None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:55.184927 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86123a-587f-4db8-af1d-5e7038a0257e None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:55.185326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86123a-587f-4db8-af1d-5e7038a0257e None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:55.185981 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86123a-587f-4db8-af1d-5e7038a0257e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:55.186288 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86123a-587f-4db8-af1d-5e7038a0257e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:55.187702 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86123a-587f-4db8-af1d-5e7038a0257e None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:55.201243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 290/564] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:55.205781 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86123a-587f-4db8-af1d-5e7038a0257e None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:55.267509 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5cdd6c-c51b-456a-831e-ec2f849e3eaf None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:55.294442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:55.304911 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd177f69-6b8c-4a27-abe9-ad265f077eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:55.305521 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd177f69-6b8c-4a27-abe9-ad265f077eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:55.305792 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd177f69-6b8c-4a27-abe9-ad265f077eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:55.306060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd177f69-6b8c-4a27-abe9-ad265f077eb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:55.306720 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd177f69-6b8c-4a27-abe9-ad265f077eb1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:55.307074 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd177f69-6b8c-4a27-abe9-ad265f077eb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:55.308412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd177f69-6b8c-4a27-abe9-ad265f077eb1 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:55.317798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 275/565] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 31 00:07:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:55.318726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86123a-587f-4db8-af1d-5e7038a0257e None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:55.320582 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8206b-63ca-48f3-b710-98b7f8e2eb48 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:55.322255 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8206b-63ca-48f3-b710-98b7f8e2eb48 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:55.322739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8206b-63ca-48f3-b710-98b7f8e2eb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:55.343709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-31a8206b-63ca-48f3-b710-98b7f8e2eb48 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:55.344897 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-31a8206b-63ca-48f3-b710-98b7f8e2eb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:55.356831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557870cf-58e8-40e5-ac6a-73c8248d66fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:55.357748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557870cf-58e8-40e5-ac6a-73c8248d66fe None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:55.358029 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557870cf-58e8-40e5-ac6a-73c8248d66fe None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:55.358302 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557870cf-58e8-40e5-ac6a-73c8248d66fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:55.366584 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557870cf-58e8-40e5-ac6a-73c8248d66fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:55.368532 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557870cf-58e8-40e5-ac6a-73c8248d66fe None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:55.383729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-557870cf-58e8-40e5-ac6a-73c8248d66fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:07:55.383729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:55.383729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:55.383729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:55.383729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:55.383729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:55.383729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:55.383729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:55.383729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:55.383729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:55.383729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:55.383729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:55.383729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:07:55.383729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:07:55.383729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:07:55.384616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:07:55.384616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:55.384616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:55.384616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:55.384616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:55.384616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:55.384616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:55.384616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:55.384616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:55.384616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:55.384616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:55.384616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:55.384616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:55.384616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:55.384616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:55.384616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:55.384616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:55.385485 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:07:55.385485 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:07:55.385485 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:07:55.385485 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:07:55.385485 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:07:55.385485 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:07:55.385485 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:07:55.385485 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:55.387179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 291/566] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:07:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:55.388916 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557870cf-58e8-40e5-ac6a-73c8248d66fe None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:55.419712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 276/567] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:55.420560 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8206b-63ca-48f3-b710-98b7f8e2eb48 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:55.432813 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:55.506461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557870cf-58e8-40e5-ac6a-73c8248d66fe None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:55.527939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8206b-63ca-48f3-b710-98b7f8e2eb48 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:55.542313 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0eb1a-bc8f-4b1e-b09e-ef78be5ef845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:55.542946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0eb1a-bc8f-4b1e-b09e-ef78be5ef845 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:55.543256 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0eb1a-bc8f-4b1e-b09e-ef78be5ef845 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:55.544365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0eb1a-bc8f-4b1e-b09e-ef78be5ef845 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:55.548795 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:55.552734 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed0eb1a-bc8f-4b1e-b09e-ef78be5ef845 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:55.554193 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed0eb1a-bc8f-4b1e-b09e-ef78be5ef845 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:55.565995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-7ed0eb1a-bc8f-4b1e-b09e-ef78be5ef845 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:07:55.565995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:55.565995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:55.565995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:55.565995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:55.565995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:55.565995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:55.565995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:55.565995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:55.565995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:55.565995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:55.565995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:55.565995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:07:55.565995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:07:55.565995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:07:55.565995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:07:55.566929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:55.566929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:55.566929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:07:55.566929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:07:55.566929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:07:55.566929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:07:55.566929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:55.566929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:55.566929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:55.566929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:55.566929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:55.566929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:55.566929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:55.566929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:55.566929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:55.566929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:55.566929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:55.566929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:55.568476 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:55.568476 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:55.568476 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:07:55.568476 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:07:55.568476 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:07:55.568476 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:07:55.568476 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:07:55.568476 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:07:55.568476 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:07:55.568476 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:55.569003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3db09d-53bf-4b9b-97bd-5e4a8e3e5cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:55.569003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3db09d-53bf-4b9b-97bd-5e4a8e3e5cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:55.569003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3db09d-53bf-4b9b-97bd-5e4a8e3e5cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:55.569003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3db09d-53bf-4b9b-97bd-5e4a8e3e5cd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:55.571913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 292/568] 10.222.0.67 () {56 vars in 1093 bytes} [Wed Aug 31 00:07:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:55.572491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0eb1a-bc8f-4b1e-b09e-ef78be5ef845 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:55.579831 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3db09d-53bf-4b9b-97bd-5e4a8e3e5cd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:55.581284 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3db09d-53bf-4b9b-97bd-5e4a8e3e5cd0 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:55.596572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-6a3db09d-53bf-4b9b-97bd-5e4a8e3e5cd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:55.596572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:55.596572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:55.596572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:55.596572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:55.596572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:55.596572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:55.596572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:55.596572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:55.596572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:55.596572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:55.596572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:55.596572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:07:55.596572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:07:55.596572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:07:55.596572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:07:55.597497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:55.597497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:55.597497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:55.597497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:55.597497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:55.597497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:55.597497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:55.597497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:55.597497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:55.597497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:55.597497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:55.597497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:55.597497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:55.597497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:55.597497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:55.597497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:55.597497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:07:55.598405 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:07:55.598405 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:55.598405 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:55.598405 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 277/569] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:55.598405 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3db09d-53bf-4b9b-97bd-5e4a8e3e5cd0 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:55.683349 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:55.687394 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0eb1a-bc8f-4b1e-b09e-ef78be5ef845 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:55.708989 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3db09d-53bf-4b9b-97bd-5e4a8e3e5cd0 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:55.722315 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3fc9a-97fd-41dc-b3e0-bd93d1769f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:55.722929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3fc9a-97fd-41dc-b3e0-bd93d1769f5d None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:55.723251 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3fc9a-97fd-41dc-b3e0-bd93d1769f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:55.723525 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3fc9a-97fd-41dc-b3e0-bd93d1769f5d None admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:55.724224 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc3fc9a-97fd-41dc-b3e0-bd93d1769f5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:55.724532 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc3fc9a-97fd-41dc-b3e0-bd93d1769f5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:55.725968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc3fc9a-97fd-41dc-b3e0-bd93d1769f5d None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:55.738880 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 293/570] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:07:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:55.740306 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3fc9a-97fd-41dc-b3e0-bd93d1769f5d None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:55.754396 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e4d94-ad32-423c-90c4-1ce38cc70b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:55.755061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e4d94-ad32-423c-90c4-1ce38cc70b28 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:55.755330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e4d94-ad32-423c-90c4-1ce38cc70b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:55.755605 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e4d94-ad32-423c-90c4-1ce38cc70b28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:55.766258 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e4d94-ad32-423c-90c4-1ce38cc70b28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:55.771563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-ef3e4d94-ad32-423c-90c4-1ce38cc70b28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:07:55.771563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:55.771563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:55.771563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:55.771563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:55.771563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:55.771563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:55.771563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:55.771563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:55.771563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:55.771563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:55.771563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:55.771563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:55.771563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:55.771563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:55.771563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:55.772648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:55.772648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:55.772648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:55.772648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:55.772648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:55.772648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:55.772648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:55.772648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:55.772648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:55.772648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:55.772648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:55.772648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:55.772648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:55.772648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:55.772648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:55.772648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:55.774059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:55.774059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:55.774059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:55.774059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:55.774059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:55.774059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:55.774059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:55.774059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:55.774059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:55.774059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:55.774059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:07:55.774059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:55.774059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 278/571] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 31 00:07:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:55.774059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e4d94-ad32-423c-90c4-1ce38cc70b28 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:55.805155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:55.851563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3fc9a-97fd-41dc-b3e0-bd93d1769f5d None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:55.880489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e4d94-ad32-423c-90c4-1ce38cc70b28 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:55.886117 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba43553-4848-43a3-9921-e2470e88cc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:55.886731 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba43553-4848-43a3-9921-e2470e88cc00 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:55.887010 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba43553-4848-43a3-9921-e2470e88cc00 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:55.887318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba43553-4848-43a3-9921-e2470e88cc00 None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:55.888795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba43553-4848-43a3-9921-e2470e88cc00 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:55.889009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba43553-4848-43a3-9921-e2470e88cc00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:55.890444 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba43553-4848-43a3-9921-e2470e88cc00 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:55.908914 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 294/572] 10.222.0.67 () {56 vars in 1133 bytes} [Wed Aug 31 00:07:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:55.909726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba43553-4848-43a3-9921-e2470e88cc00 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:55.915503 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4930025-58ba-4f9b-816d-b7664659ee1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:55.916118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4930025-58ba-4f9b-816d-b7664659ee1a None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:55.916399 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4930025-58ba-4f9b-816d-b7664659ee1a None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:55.916671 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4930025-58ba-4f9b-816d-b7664659ee1a None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:55.917315 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4930025-58ba-4f9b-816d-b7664659ee1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:55.917626 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4930025-58ba-4f9b-816d-b7664659ee1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:55.919126 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4930025-58ba-4f9b-816d-b7664659ee1a None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:55.928684 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:55.931953 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 279/573] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:55.932582 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4930025-58ba-4f9b-816d-b7664659ee1a None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:56.013403 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba43553-4848-43a3-9921-e2470e88cc00 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:56.027098 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:56.037463 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4930025-58ba-4f9b-816d-b7664659ee1a None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:56.049255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bae25b-7f8f-4365-ad55-11fd1d0c43d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:56.049887 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bae25b-7f8f-4365-ad55-11fd1d0c43d1 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:56.050165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bae25b-7f8f-4365-ad55-11fd1d0c43d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:56.050434 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bae25b-7f8f-4365-ad55-11fd1d0c43d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:56.061199 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bae25b-7f8f-4365-ad55-11fd1d0c43d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:56.067547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-e5bae25b-7f8f-4365-ad55-11fd1d0c43d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:07:56.067547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:56.067547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:56.067547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:56.067547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:56.067547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:56.067547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:56.067547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:56.067547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:56.067547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:56.067547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:56.067547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:56.067547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:56.067547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:56.067547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:56.067547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:56.068673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:56.068673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:56.068673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:56.068673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:56.068673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:56.068673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:56.068673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:56.068673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:56.068673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:56.068673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:56.068673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:56.068673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:56.068673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:56.068673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:56.068673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:56.068673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:56.069705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:56.069705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:56.069705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:56.069705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:56.069705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:56.069705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:56.069705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:56.069705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:56.069705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:56.069705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:56.069705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:07:56.069705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:56.075626 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 295/574] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 31 00:07:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:56.076364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b4e31-e700-42f3-bad3-639b80326803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:56.076860 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bae25b-7f8f-4365-ad55-11fd1d0c43d1 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:56.076976 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b4e31-e700-42f3-bad3-639b80326803 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:56.077233 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b4e31-e700-42f3-bad3-639b80326803 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:56.077507 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b4e31-e700-42f3-bad3-639b80326803 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:56.078226 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b4e31-e700-42f3-bad3-639b80326803 None admin] RBAC: Adding query filter params (role.id=4680aa64da80470b87075090fca2354b, scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb, user.id=1cd3cfbc8ebd4c5a83bef09efe10bc91) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:56.078529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b4e31-e700-42f3-bad3-639b80326803 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:56.079972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b4e31-e700-42f3-bad3-639b80326803 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:56.085587 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 280/575] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:07:55 2022] GET /identity/v3/role_assignments?user.id=1cd3cfbc8ebd4c5a83bef09efe10bc91&scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb&role.id=4680aa64da80470b87075090fca2354b => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:56.086363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b4e31-e700-42f3-bad3-639b80326803 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:56.179909 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:56.204103 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bae25b-7f8f-4365-ad55-11fd1d0c43d1 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:56.210123 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b4e31-e700-42f3-bad3-639b80326803 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:56.242555 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d555711-31e1-4095-8ed6-c87b0f31d358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:56.243210 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d555711-31e1-4095-8ed6-c87b0f31d358 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:56.243485 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d555711-31e1-4095-8ed6-c87b0f31d358 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:56.243753 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d555711-31e1-4095-8ed6-c87b0f31d358 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:56.244412 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d555711-31e1-4095-8ed6-c87b0f31d358 None admin] RBAC: Adding query filter params (name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:56.244713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d555711-31e1-4095-8ed6-c87b0f31d358 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:56.246142 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d555711-31e1-4095-8ed6-c87b0f31d358 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:56.250711 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528e5e9-7bc7-48b7-8b8e-4ec66bbb23be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:56.251383 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528e5e9-7bc7-48b7-8b8e-4ec66bbb23be None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:56.251661 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528e5e9-7bc7-48b7-8b8e-4ec66bbb23be None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:56.251925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528e5e9-7bc7-48b7-8b8e-4ec66bbb23be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:56.259794 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 296/576] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 31 00:07:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:56.259794 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d555711-31e1-4095-8ed6-c87b0f31d358 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:56.263240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528e5e9-7bc7-48b7-8b8e-4ec66bbb23be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:56.264678 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528e5e9-7bc7-48b7-8b8e-4ec66bbb23be None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:56.280622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-e528e5e9-7bc7-48b7-8b8e-4ec66bbb23be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:56.280622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:56.280622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:56.280622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:56.280622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:56.280622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:56.280622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:56.280622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:56.280622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:56.280622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:56.280622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:56.280622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:56.280622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:07:56.280622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:07:56.280622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:07:56.280622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:07:56.281631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:56.281631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:56.281631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:56.281631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:56.281631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:56.281631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:56.281631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:56.281631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:56.281631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:56.281631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:56.281631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:56.281631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:56.281631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:56.281631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:56.281631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:56.281631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:56.281631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:07:56.287739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:07:56.287739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:56.287739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:56.287739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 281/577] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:56.287739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528e5e9-7bc7-48b7-8b8e-4ec66bbb23be None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:56.291789 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:56.378198 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d555711-31e1-4095-8ed6-c87b0f31d358 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:56.405105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528e5e9-7bc7-48b7-8b8e-4ec66bbb23be None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:56.420927 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72bf8ef-96ca-496b-9db8-e73db1fff96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:56.421566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bf8ef-96ca-496b-9db8-e73db1fff96a None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:56.421849 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bf8ef-96ca-496b-9db8-e73db1fff96a None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:56.422123 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bf8ef-96ca-496b-9db8-e73db1fff96a None admin] PATH_INFO: `/v3/users/glance` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:56.430412 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72bf8ef-96ca-496b-9db8-e73db1fff96a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:56.431893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72bf8ef-96ca-496b-9db8-e73db1fff96a None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:56.435001 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:56.444141 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0e276-1458-4916-9730-82e1ac53e0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:56.444752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0e276-1458-4916-9730-82e1ac53e0f7 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:56.444854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-f72bf8ef-96ca-496b-9db8-e73db1fff96a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 00:07:56.444854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:56.444854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:56.444854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:56.444854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:56.444854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:56.444854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:56.444854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:56.444854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:56.444854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:56.444854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:56.444854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:56.444854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:07:56.444854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:07:56.444854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:07:56.444854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:07:56.445726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:56.445726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:56.445726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:07:56.445726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:07:56.445726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:07:56.445726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:07:56.445726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:56.445726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:56.445726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:56.445726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:56.445726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:56.445726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:56.445726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:56.445726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:56.445726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:56.445726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:56.445726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:56.445726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:56.446676 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:56.446676 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:56.446676 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:07:56.446676 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:07:56.446676 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:07:56.446676 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:07:56.446676 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:07:56.446676 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:07:56.446676 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 00:07:56.446676 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:56.447224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0e276-1458-4916-9730-82e1ac53e0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:56.447224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0e276-1458-4916-9730-82e1ac53e0f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:56.447346 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 297/578] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:56.447346 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72bf8ef-96ca-496b-9db8-e73db1fff96a None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:56.452053 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de0e276-1458-4916-9730-82e1ac53e0f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:56.452053 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de0e276-1458-4916-9730-82e1ac53e0f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:56.453411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de0e276-1458-4916-9730-82e1ac53e0f7 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:56.462312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 282/579] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 31 00:07:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:56.464002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0e276-1458-4916-9730-82e1ac53e0f7 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:56.540095 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:56.562170 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72bf8ef-96ca-496b-9db8-e73db1fff96a None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:56.575458 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0e276-1458-4916-9730-82e1ac53e0f7 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:56.599255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f96f0-f41f-4f45-90ac-94ac6cc6669d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:56.599904 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f96f0-f41f-4f45-90ac-94ac6cc6669d None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:56.600183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f96f0-f41f-4f45-90ac-94ac6cc6669d None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:56.600456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f96f0-f41f-4f45-90ac-94ac6cc6669d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:56.609574 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51868c15-7a12-4a69-9a1c-383fe3a772f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:56.610184 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51868c15-7a12-4a69-9a1c-383fe3a772f2 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:56.610464 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51868c15-7a12-4a69-9a1c-383fe3a772f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:56.610731 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51868c15-7a12-4a69-9a1c-383fe3a772f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:56.611397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51868c15-7a12-4a69-9a1c-383fe3a772f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:56.611698 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51868c15-7a12-4a69-9a1c-383fe3a772f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:56.612791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f96f0-f41f-4f45-90ac-94ac6cc6669d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:56.613049 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51868c15-7a12-4a69-9a1c-383fe3a772f2 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:56.614727 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f96f0-f41f-4f45-90ac-94ac6cc6669d None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:56.625647 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 283/580] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:07:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:56.626461 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51868c15-7a12-4a69-9a1c-383fe3a772f2 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:56.630330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-332f96f0-f41f-4f45-90ac-94ac6cc6669d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:56.630330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:56.630330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:56.630330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:56.630330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:56.630330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:56.630330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:56.630330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:56.630330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:56.630330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:56.630330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:56.630330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:56.630330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:07:56.630330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:07:56.630330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:07:56.630330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:07:56.631393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:56.631393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:56.631393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:56.631393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:56.631393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:56.631393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:56.631393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:56.631393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:56.631393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:56.631393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:56.631393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:56.631393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:56.631393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:56.631393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:56.631393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:56.631393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:56.631393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:07:56.632257 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:07:56.632257 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:56.632257 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:56.632421 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 298/581] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:56.633082 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f96f0-f41f-4f45-90ac-94ac6cc6669d None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:56.688674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:56.743909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51868c15-7a12-4a69-9a1c-383fe3a772f2 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:56.751059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f96f0-f41f-4f45-90ac-94ac6cc6669d None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:56.772635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:56.782190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d5b53-06b8-4231-b9a9-b14be9e33985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:56.782816 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d5b53-06b8-4231-b9a9-b14be9e33985 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:56.783095 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d5b53-06b8-4231-b9a9-b14be9e33985 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:56.783369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d5b53-06b8-4231-b9a9-b14be9e33985 None admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:56.784088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505d5b53-06b8-4231-b9a9-b14be9e33985 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:56.784394 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505d5b53-06b8-4231-b9a9-b14be9e33985 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:56.785793 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505d5b53-06b8-4231-b9a9-b14be9e33985 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:56.790437 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0f0ae-1e30-4ed3-b45c-0f43c561b26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:56.791269 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0f0ae-1e30-4ed3-b45c-0f43c561b26d None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:56.791696 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0f0ae-1e30-4ed3-b45c-0f43c561b26d None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:56.792135 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0f0ae-1e30-4ed3-b45c-0f43c561b26d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:56.803965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0f0ae-1e30-4ed3-b45c-0f43c561b26d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:56.808553 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 284/582] 10.222.0.67 () {56 vars in 1137 bytes} [Wed Aug 31 00:07:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:56.808553 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d5b53-06b8-4231-b9a9-b14be9e33985 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:56.808749 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0f0ae-1e30-4ed3-b45c-0f43c561b26d None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:56.824329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-82d0f0ae-1e30-4ed3-b45c-0f43c561b26d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:56.824329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:56.824329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:56.824329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:56.824329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:56.824329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:56.824329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:56.824329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:56.824329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:56.824329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:56.824329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:56.824329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:56.824329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:07:56.824329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:07:56.824329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:07:56.824329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:07:56.826111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:56.826111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:56.826111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:56.826111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:56.826111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:56.826111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:56.826111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:56.826111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:56.826111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:56.826111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:56.826111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:56.826111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:56.826111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:56.826111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:56.826111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:56.826111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:56.826111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:07:56.826990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:07:56.826990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:56.826990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:56.832776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 299/583] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:56.832776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0f0ae-1e30-4ed3-b45c-0f43c561b26d None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:56.922670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d5b53-06b8-4231-b9a9-b14be9e33985 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:56.940877 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0f0ae-1e30-4ed3-b45c-0f43c561b26d None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:56.944074 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:56.964451 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83141a-e6fd-42ab-a0f9-a323262c5943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:56.965086 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83141a-e6fd-42ab-a0f9-a323262c5943 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:56.965367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83141a-e6fd-42ab-a0f9-a323262c5943 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:56.965757 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83141a-e6fd-42ab-a0f9-a323262c5943 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:56.973733 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83141a-e6fd-42ab-a0f9-a323262c5943 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:56.975211 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83141a-e6fd-42ab-a0f9-a323262c5943 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:56.982202 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318cd50-21b0-431d-a6af-253be772ce96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:56.982836 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318cd50-21b0-431d-a6af-253be772ce96 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:56.983135 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318cd50-21b0-431d-a6af-253be772ce96 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:56.983407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318cd50-21b0-431d-a6af-253be772ce96 None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:56.984107 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318cd50-21b0-431d-a6af-253be772ce96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:56.984412 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318cd50-21b0-431d-a6af-253be772ce96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:56.985766 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318cd50-21b0-431d-a6af-253be772ce96 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:56.986908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-1a83141a-e6fd-42ab-a0f9-a323262c5943 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:07:56.986908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:56.986908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:56.986908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:56.986908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:56.986908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:56.986908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:56.986908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:56.986908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:56.986908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:56.986908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:56.986908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:56.986908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:07:56.986908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:07:56.986908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:07:56.987806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:07:56.987806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:56.987806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:56.987806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:56.987806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:56.987806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:56.987806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:56.987806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:56.987806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:56.987806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:56.987806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:56.987806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:56.987806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:56.987806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:56.987806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:56.987806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:56.987806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:56.988696 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:07:56.988696 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:07:56.988696 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:07:56.988696 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:07:56.988696 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:07:56.988696 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:07:56.988696 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:07:56.988696 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:56.988696 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 285/584] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:07:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:56.989712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83141a-e6fd-42ab-a0f9-a323262c5943 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:57.004590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 300/585] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:07:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:57.005246 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318cd50-21b0-431d-a6af-253be772ce96 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:57.048167 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:57.110735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83141a-e6fd-42ab-a0f9-a323262c5943 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:57.120443 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318cd50-21b0-431d-a6af-253be772ce96 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:57.149396 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630b394-7cb4-4adc-9b70-a340498b6889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:57.150029 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630b394-7cb4-4adc-9b70-a340498b6889 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:57.150313 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630b394-7cb4-4adc-9b70-a340498b6889 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:57.150590 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630b394-7cb4-4adc-9b70-a340498b6889 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:57.158178 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b77348-1446-4fe5-84c2-c05ac8fa8b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:57.158786 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b77348-1446-4fe5-84c2-c05ac8fa8b59 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:57.159095 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b77348-1446-4fe5-84c2-c05ac8fa8b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:57.159374 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b77348-1446-4fe5-84c2-c05ac8fa8b59 None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:57.160015 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b77348-1446-4fe5-84c2-c05ac8fa8b59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:57.160322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b77348-1446-4fe5-84c2-c05ac8fa8b59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:57.161694 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b77348-1446-4fe5-84c2-c05ac8fa8b59 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:57.161931 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630b394-7cb4-4adc-9b70-a340498b6889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:57.163390 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630b394-7cb4-4adc-9b70-a340498b6889 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:57.178819 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 301/586] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:57.178819 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b77348-1446-4fe5-84c2-c05ac8fa8b59 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:57.179495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-f630b394-7cb4-4adc-9b70-a340498b6889 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:57.179495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:57.179495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:57.179495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:57.179495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:57.179495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:57.179495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:57.179495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:57.179495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:57.179495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:57.179495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:57.179495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:57.179495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:07:57.179495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:07:57.179495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:07:57.179495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:07:57.180392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:57.180392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:57.180392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:57.180392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:57.180392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:57.180392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:57.180392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:57.180392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:57.180392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:57.180392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:57.180392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:57.180392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:57.180392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:57.180392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:57.180392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:57.180392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:57.180392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:07:57.181267 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:07:57.181267 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:57.181267 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:57.181267 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 286/587] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:57.181891 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630b394-7cb4-4adc-9b70-a340498b6889 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:57.210876 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:57.292586 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b77348-1446-4fe5-84c2-c05ac8fa8b59 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:57.299878 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:57.305786 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630b394-7cb4-4adc-9b70-a340498b6889 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:57.329935 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad5d66-cde2-446e-a394-7a7e89ce8255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:57.330547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad5d66-cde2-446e-a394-7a7e89ce8255 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:57.330828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad5d66-cde2-446e-a394-7a7e89ce8255 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:57.331142 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad5d66-cde2-446e-a394-7a7e89ce8255 None admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:57.331837 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad5d66-cde2-446e-a394-7a7e89ce8255 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:57.332149 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad5d66-cde2-446e-a394-7a7e89ce8255 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:57.333577 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad5d66-cde2-446e-a394-7a7e89ce8255 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:57.341994 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2a8e3-bcc4-4817-ac13-f8de3495c853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:57.342823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2a8e3-bcc4-4817-ac13-f8de3495c853 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:57.343289 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2a8e3-bcc4-4817-ac13-f8de3495c853 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:57.343561 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2a8e3-bcc4-4817-ac13-f8de3495c853 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:57.346641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 302/588] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:07:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:57.347503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad5d66-cde2-446e-a394-7a7e89ce8255 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:57.353680 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a2a8e3-bcc4-4817-ac13-f8de3495c853 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:57.355245 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a2a8e3-bcc4-4817-ac13-f8de3495c853 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:57.370343 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-14a2a8e3-bcc4-4817-ac13-f8de3495c853 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 00:07:57.370343 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:57.370343 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:57.370343 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:57.370343 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:57.370343 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:57.370343 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:57.370343 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:57.370343 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:57.370343 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:57.370343 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:57.370343 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:57.370343 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:07:57.370343 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:07:57.370343 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:07:57.370343 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:07:57.379264 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:57.379264 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:57.379264 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:07:57.379264 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:07:57.379264 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:07:57.379264 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:07:57.379264 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:57.379264 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:57.379264 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:57.379264 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:57.379264 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:57.379264 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:57.379264 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:57.379264 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:57.379264 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:57.379264 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:57.379264 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:57.379264 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:57.381208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:57.381208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:57.381208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:07:57.381208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:07:57.381208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:07:57.381208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:07:57.381208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:07:57.381208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:07:57.381208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 00:07:57.381208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:57.381208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 287/589] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:07:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:57.381208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2a8e3-bcc4-4817-ac13-f8de3495c853 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:57.449716 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:57.466539 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad5d66-cde2-446e-a394-7a7e89ce8255 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:57.493558 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2a8e3-bcc4-4817-ac13-f8de3495c853 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:57.502558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7ca55-2afb-419a-8f28-fba62bb6165b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:57.503223 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7ca55-2afb-419a-8f28-fba62bb6165b None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:57.503507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7ca55-2afb-419a-8f28-fba62bb6165b None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:57.503779 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7ca55-2afb-419a-8f28-fba62bb6165b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:57.511077 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7ca55-2afb-419a-8f28-fba62bb6165b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:57.513190 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7ca55-2afb-419a-8f28-fba62bb6165b None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:57.525466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-70c7ca55-2afb-419a-8f28-fba62bb6165b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 00:07:57.525466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:57.525466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:57.525466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:57.525466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:57.525466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:57.525466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:57.525466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:57.525466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:57.525466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:57.525466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:57.525466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:57.525466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:07:57.525466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:07:57.525466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:07:57.525466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:07:57.526415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:57.526415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:57.526415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:07:57.526415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:07:57.526415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:07:57.526415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:07:57.526415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:57.526415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:57.526415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:57.526415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:57.526415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:57.526415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:57.526415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:57.526415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:57.526415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:57.526415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:57.526415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:57.526415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:57.527404 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:57.527404 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:57.527404 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:07:57.527404 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:07:57.527404 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:07:57.527404 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:07:57.527404 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:07:57.527404 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:07:57.527404 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 00:07:57.527404 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:57.527404 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 303/590] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 31 00:07:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:57.527404 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7ca55-2afb-419a-8f28-fba62bb6165b None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:57.536507 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e898bb4-56b0-4b53-8720-1968a88c51af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:57.537174 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e898bb4-56b0-4b53-8720-1968a88c51af None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:57.537460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e898bb4-56b0-4b53-8720-1968a88c51af None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:57.537742 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e898bb4-56b0-4b53-8720-1968a88c51af None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:57.538387 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e898bb4-56b0-4b53-8720-1968a88c51af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:57.538691 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e898bb4-56b0-4b53-8720-1968a88c51af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:57.540079 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e898bb4-56b0-4b53-8720-1968a88c51af None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:57.557204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 288/591] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:57.557204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e898bb4-56b0-4b53-8720-1968a88c51af None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:57.573092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:57.643248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7ca55-2afb-419a-8f28-fba62bb6165b None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:57.660874 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e898bb4-56b0-4b53-8720-1968a88c51af None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:57.678543 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905f166-e726-4056-a7dd-69ed78557e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:57.679248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905f166-e726-4056-a7dd-69ed78557e9f None admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:57.679536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905f166-e726-4056-a7dd-69ed78557e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:57.679814 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905f166-e726-4056-a7dd-69ed78557e9f None admin] PATH_INFO: `/v3/projects/cd1e6daca4264a0cbc11a9160d1213eb/users/3aa53ae5a30146f69d7d6eefe4290dba/roles/4680aa64da80470b87075090fca2354b` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:57.696650 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e905f166-e726-4056-a7dd-69ed78557e9f None admin] RBAC: Authorizing `identity:create_grant(project_id=cd1e6daca4264a0cbc11a9160d1213eb, user_id=3aa53ae5a30146f69d7d6eefe4290dba, role_id=4680aa64da80470b87075090fca2354b)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:57.698110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e905f166-e726-4056-a7dd-69ed78557e9f None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:57.698943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09babcd3-cbf9-4757-b565-5784f3ea6612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:57.699638 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09babcd3-cbf9-4757-b565-5784f3ea6612 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:57.700364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09babcd3-cbf9-4757-b565-5784f3ea6612 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:57.700364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09babcd3-cbf9-4757-b565-5784f3ea6612 None admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:57.701079 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09babcd3-cbf9-4757-b565-5784f3ea6612 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:57.701224 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:57.701387 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09babcd3-cbf9-4757-b565-5784f3ea6612 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:57.702873 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09babcd3-cbf9-4757-b565-5784f3ea6612 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:57.712700 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 304/592] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 31 00:07:57 2022] PUT /identity/v3/projects/cd1e6daca4264a0cbc11a9160d1213eb/users/3aa53ae5a30146f69d7d6eefe4290dba/roles/4680aa64da80470b87075090fca2354b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:07:57.713718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905f166-e726-4056-a7dd-69ed78557e9f None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:57.737894 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 289/593] 10.222.0.67 () {56 vars in 1137 bytes} [Wed Aug 31 00:07:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:57.739986 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09babcd3-cbf9-4757-b565-5784f3ea6612 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:57.849396 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:57.868312 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905f166-e726-4056-a7dd-69ed78557e9f None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:57.880564 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09babcd3-cbf9-4757-b565-5784f3ea6612 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:57.911310 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8441ebd-812d-4c1c-a1a3-a9e9b028a8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:57.912315 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8441ebd-812d-4c1c-a1a3-a9e9b028a8de None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:57.912315 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8441ebd-812d-4c1c-a1a3-a9e9b028a8de None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:57.912513 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8441ebd-812d-4c1c-a1a3-a9e9b028a8de None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:57.913178 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8441ebd-812d-4c1c-a1a3-a9e9b028a8de None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:57.913484 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8441ebd-812d-4c1c-a1a3-a9e9b028a8de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:57.915110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8441ebd-812d-4c1c-a1a3-a9e9b028a8de None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:57.920672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39039d4a-26c7-42d1-a0b7-b7180b603606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:57.921444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39039d4a-26c7-42d1-a0b7-b7180b603606 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:57.921817 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39039d4a-26c7-42d1-a0b7-b7180b603606 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:57.922182 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39039d4a-26c7-42d1-a0b7-b7180b603606 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:57.932548 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39039d4a-26c7-42d1-a0b7-b7180b603606 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:57.934043 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 305/594] 10.222.0.67 () {56 vars in 1139 bytes} [Wed Aug 31 00:07:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:57.935114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39039d4a-26c7-42d1-a0b7-b7180b603606 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:57.935887 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8441ebd-812d-4c1c-a1a3-a9e9b028a8de None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:57.950211 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-39039d4a-26c7-42d1-a0b7-b7180b603606 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:07:57.950211 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:57.950211 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:57.950211 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:57.950211 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:57.950211 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:57.950211 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:57.950211 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:57.950211 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:57.950211 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:57.950211 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:57.950211 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:57.950211 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:07:57.950211 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:07:57.950211 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:07:57.951397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:07:57.951397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:57.951397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:57.951397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:57.951397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:57.951397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:57.951397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:57.951397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:57.951397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:57.951397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:57.951397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:57.951397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:57.951397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:57.951397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:57.951397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:57.951397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:57.951397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:57.952444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:07:57.952444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:07:57.952444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:07:57.952444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:07:57.952444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:07:57.952444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:07:57.952444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:07:57.952444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:57.952444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 290/595] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:07:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:57.953430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39039d4a-26c7-42d1-a0b7-b7180b603606 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:58.015463 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:58.062397 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8441ebd-812d-4c1c-a1a3-a9e9b028a8de None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:58.077896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39039d4a-26c7-42d1-a0b7-b7180b603606 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:58.098029 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc8a1a-2e0a-41ae-a5f1-de3f34ee02fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:58.098667 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc8a1a-2e0a-41ae-a5f1-de3f34ee02fa None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:58.098939 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc8a1a-2e0a-41ae-a5f1-de3f34ee02fa None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:58.099235 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc8a1a-2e0a-41ae-a5f1-de3f34ee02fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:58.110845 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc8a1a-2e0a-41ae-a5f1-de3f34ee02fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:58.112331 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc8a1a-2e0a-41ae-a5f1-de3f34ee02fa None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:58.120650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d9a186-be94-44f5-8ee1-6811f1cfb9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:58.121848 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d9a186-be94-44f5-8ee1-6811f1cfb9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:58.123024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d9a186-be94-44f5-8ee1-6811f1cfb9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:58.123342 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d9a186-be94-44f5-8ee1-6811f1cfb9d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:58.126905 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:58.128500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-efbc8a1a-2e0a-41ae-a5f1-de3f34ee02fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:58.128500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:58.128500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:58.128500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:58.128500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:58.128500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:58.128500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:58.128500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:58.128500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:58.128500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:58.128500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:58.128500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:58.128500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:07:58.128500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:07:58.128500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:07:58.128500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:07:58.129629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:58.129629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:58.129629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:58.129629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:58.129629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:58.129629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:58.129629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:58.129629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:58.129629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:58.129629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:58.129629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:58.129629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:58.129629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:58.129629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:58.129629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:58.129629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:58.129629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:07:58.130948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:07:58.130948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:58.130948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:58.135246 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 306/596] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:58.136081 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc8a1a-2e0a-41ae-a5f1-de3f34ee02fa None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:58.136185 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d9a186-be94-44f5-8ee1-6811f1cfb9d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:58.138327 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d9a186-be94-44f5-8ee1-6811f1cfb9d4 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:58.155709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-51d9a186-be94-44f5-8ee1-6811f1cfb9d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:58.155709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:58.155709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:58.155709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:58.155709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:58.155709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:58.155709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:58.155709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:58.155709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:58.155709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:58.155709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:58.155709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:58.155709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:07:58.155709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:07:58.155709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:07:58.155709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:07:58.156872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:58.156872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:58.156872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:58.156872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:58.156872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:58.156872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:58.156872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:58.156872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:58.156872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:58.156872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:58.156872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:58.156872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:58.156872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:58.156872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:58.156872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:58.156872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:58.156872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:07:58.158037 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:07:58.158037 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:58.158037 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:58.161851 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 291/597] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:58.162719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263d8aa-7338-4eb5-94c4-dadce836a8da None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:58.163005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263d8aa-7338-4eb5-94c4-dadce836a8da None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:58.163302 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263d8aa-7338-4eb5-94c4-dadce836a8da None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:58.167080 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 292/598] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:07:58.167080 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263d8aa-7338-4eb5-94c4-dadce836a8da None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:58.251749 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc8a1a-2e0a-41ae-a5f1-de3f34ee02fa None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:58.254806 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:58.286512 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263d8aa-7338-4eb5-94c4-dadce836a8da None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:58.287873 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7766a6b-5991-45ba-a292-c2983e9d3fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:58.288641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7766a6b-5991-45ba-a292-c2983e9d3fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:58.289710 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7766a6b-5991-45ba-a292-c2983e9d3fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:58.289992 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7766a6b-5991-45ba-a292-c2983e9d3fb5 None admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:58.290688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7766a6b-5991-45ba-a292-c2983e9d3fb5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:58.290994 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7766a6b-5991-45ba-a292-c2983e9d3fb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:58.292454 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7766a6b-5991-45ba-a292-c2983e9d3fb5 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:58.312402 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 307/599] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:07:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:58.312402 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7766a6b-5991-45ba-a292-c2983e9d3fb5 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:58.324026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34244a29-6df5-491d-b49e-d5cf91aac881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:58.324644 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34244a29-6df5-491d-b49e-d5cf91aac881 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:58.325655 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34244a29-6df5-491d-b49e-d5cf91aac881 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:58.326034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34244a29-6df5-491d-b49e-d5cf91aac881 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:58.326775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34244a29-6df5-491d-b49e-d5cf91aac881 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:58.327107 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34244a29-6df5-491d-b49e-d5cf91aac881 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:58.328468 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34244a29-6df5-491d-b49e-d5cf91aac881 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:58.345594 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 293/600] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:07:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:58.345594 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c1da4-8a2c-4746-8a38-77f868f4438c None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:58.345594 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c1da4-8a2c-4746-8a38-77f868f4438c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:58.345594 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c1da4-8a2c-4746-8a38-77f868f4438c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:58.361238 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-9d1c1da4-8a2c-4746-8a38-77f868f4438c None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:58.362414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-9d1c1da4-8a2c-4746-8a38-77f868f4438c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:58.419752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:58.438230 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7766a6b-5991-45ba-a292-c2983e9d3fb5 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:58.447222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 294/601] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:58.447944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c1da4-8a2c-4746-8a38-77f868f4438c None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:58.487492 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e7d3e-4c39-45e6-abd1-5c810ba26e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:58.488313 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e7d3e-4c39-45e6-abd1-5c810ba26e12 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:58.488798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e7d3e-4c39-45e6-abd1-5c810ba26e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:58.489272 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e7d3e-4c39-45e6-abd1-5c810ba26e12 None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:58.490025 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e7d3e-4c39-45e6-abd1-5c810ba26e12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:58.490335 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e7d3e-4c39-45e6-abd1-5c810ba26e12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:58.491749 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e7d3e-4c39-45e6-abd1-5c810ba26e12 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:58.504467 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 308/602] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:07:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:58.505140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e7d3e-4c39-45e6-abd1-5c810ba26e12 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:58.531398 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:58.561995 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c1da4-8a2c-4746-8a38-77f868f4438c None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:58.599920 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96b858-ba9c-4897-8785-ec494a9ddd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:58.600687 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96b858-ba9c-4897-8785-ec494a9ddd1a None admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:58.600970 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96b858-ba9c-4897-8785-ec494a9ddd1a None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:58.601241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96b858-ba9c-4897-8785-ec494a9ddd1a None admin] PATH_INFO: `/v3/projects/cd1e6daca4264a0cbc11a9160d1213eb/users/55714a21c1cb4f2b949603230243ed02/roles/4680aa64da80470b87075090fca2354b` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:58.609622 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e7d3e-4c39-45e6-abd1-5c810ba26e12 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:58.616977 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96b858-ba9c-4897-8785-ec494a9ddd1a None admin] RBAC: Authorizing `identity:create_grant(project_id=cd1e6daca4264a0cbc11a9160d1213eb, user_id=55714a21c1cb4f2b949603230243ed02, role_id=4680aa64da80470b87075090fca2354b)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:58.618567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96b858-ba9c-4897-8785-ec494a9ddd1a None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:58.638364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 295/603] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 31 00:07:58 2022] PUT /identity/v3/projects/cd1e6daca4264a0cbc11a9160d1213eb/users/55714a21c1cb4f2b949603230243ed02/roles/4680aa64da80470b87075090fca2354b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:07:58.639772 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96b858-ba9c-4897-8785-ec494a9ddd1a None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:58.646794 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759af24-a95d-418e-b3e0-44b2fdab674d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:58.647553 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759af24-a95d-418e-b3e0-44b2fdab674d None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:58.647834 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759af24-a95d-418e-b3e0-44b2fdab674d None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:58.648230 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759af24-a95d-418e-b3e0-44b2fdab674d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:58.657323 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759af24-a95d-418e-b3e0-44b2fdab674d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:58.658899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759af24-a95d-418e-b3e0-44b2fdab674d None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:58.672258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-3759af24-a95d-418e-b3e0-44b2fdab674d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:07:58.672258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:58.672258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:58.672258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:58.672258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:58.672258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:58.672258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:58.672258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:58.672258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:58.672258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:58.672258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:58.672258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:58.672258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:07:58.672258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:07:58.672258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:07:58.673164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:07:58.673164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:58.673164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:58.673164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:58.673164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:58.673164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:58.673164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:58.673164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:58.673164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:58.673164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:58.673164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:58.673164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:58.673164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:58.673164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:58.673164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:58.673164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:58.673164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:58.674034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:07:58.674034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:07:58.674034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:07:58.674034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:07:58.674034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:07:58.674034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:07:58.674034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:07:58.674034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:07:58.674790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:58.678325 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 309/604] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:07:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:58.679865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759af24-a95d-418e-b3e0-44b2fdab674d None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:58.776463 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96b858-ba9c-4897-8785-ec494a9ddd1a None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:58.839282 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:58.845008 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7bf19d-3ff2-4bf0-9804-a759a02d1079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:58.846176 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bf19d-3ff2-4bf0-9804-a759a02d1079 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:58.846459 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bf19d-3ff2-4bf0-9804-a759a02d1079 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:58.846773 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bf19d-3ff2-4bf0-9804-a759a02d1079 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:58.857129 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7bf19d-3ff2-4bf0-9804-a759a02d1079 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:58.858916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7bf19d-3ff2-4bf0-9804-a759a02d1079 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:58.859646 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759af24-a95d-418e-b3e0-44b2fdab674d None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:58.871900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-cd7bf19d-3ff2-4bf0-9804-a759a02d1079 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:07:58.871900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:58.871900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:58.871900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:58.871900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:58.871900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:58.871900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:58.871900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:58.871900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:58.871900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:58.871900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:58.871900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:58.871900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:07:58.871900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:07:58.871900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:07:58.874017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:07:58.874017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:58.874017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:58.874017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:58.874017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:58.874017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:58.874017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:58.874017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:58.874017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:58.874017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:58.874017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:58.874017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:58.874017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:58.874017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:58.874017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:58.874017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:58.874017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:58.875749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:07:58.875749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:07:58.875749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:07:58.875749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:07:58.875749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:07:58.875749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:07:58.875749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:07:58.875749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:58.875749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 296/605] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:07:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:58.875749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7bf19d-3ff2-4bf0-9804-a759a02d1079 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:58.912180 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702aa9d2-eb45-482e-8182-0c8662c76da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:58.912946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702aa9d2-eb45-482e-8182-0c8662c76da0 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:58.913292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702aa9d2-eb45-482e-8182-0c8662c76da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:58.913632 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702aa9d2-eb45-482e-8182-0c8662c76da0 None admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:58.914450 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702aa9d2-eb45-482e-8182-0c8662c76da0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:58.914822 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702aa9d2-eb45-482e-8182-0c8662c76da0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:58.916445 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702aa9d2-eb45-482e-8182-0c8662c76da0 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:58.930868 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 310/606] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:07:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:58.932028 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702aa9d2-eb45-482e-8182-0c8662c76da0 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:58.991943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:59.017544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7bf19d-3ff2-4bf0-9804-a759a02d1079 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:59.044415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702aa9d2-eb45-482e-8182-0c8662c76da0 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:59.063400 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd78783-fcc2-4a4c-8519-b7a45bc58c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:59.064310 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd78783-fcc2-4a4c-8519-b7a45bc58c7e None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:59.064797 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd78783-fcc2-4a4c-8519-b7a45bc58c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:59.065312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd78783-fcc2-4a4c-8519-b7a45bc58c7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:59.078610 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd78783-fcc2-4a4c-8519-b7a45bc58c7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:07:59.084642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-4fd78783-fcc2-4a4c-8519-b7a45bc58c7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:07:59.084642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:59.084642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:59.084642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:59.084642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:59.084642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:59.084642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:59.084642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:59.084642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:59.084642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:59.084642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:59.084642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:59.084642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:07:59.084642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:07:59.084642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:07:59.084642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:07:59.085568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:07:59.085568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:07:59.085568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:07:59.085568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:07:59.085568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:59.085568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:59.085568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:59.085568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:59.085568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:59.085568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:59.085568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:59.085568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:59.085568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:59.085568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:59.085568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:59.085568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:59.086424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:59.086424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:59.086424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:59.086424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:59.086424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:07:59.086424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:07:59.086424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:07:59.086424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:07:59.086424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:07:59.086424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:07:59.086424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:07:59.086424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:59.088279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac50e8-610f-47cb-96dd-122e0c674a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:59.089103 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac50e8-610f-47cb-96dd-122e0c674a2f None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:59.089261 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 297/607] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 31 00:07:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:07:59.089447 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac50e8-610f-47cb-96dd-122e0c674a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:59.089813 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac50e8-610f-47cb-96dd-122e0c674a2f None admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:59.091103 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd78783-fcc2-4a4c-8519-b7a45bc58c7e None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:59.091486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac50e8-610f-47cb-96dd-122e0c674a2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:59.092165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac50e8-610f-47cb-96dd-122e0c674a2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:59.095608 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac50e8-610f-47cb-96dd-122e0c674a2f None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:59.118729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 311/608] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:07:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:59.118729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac50e8-610f-47cb-96dd-122e0c674a2f None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:59.119329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:59.221059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd78783-fcc2-4a4c-8519-b7a45bc58c7e None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:59.247920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac50e8-610f-47cb-96dd-122e0c674a2f None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:59.267374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2af82-92bc-4e04-99a2-208bed4d5010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:59.268134 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2af82-92bc-4e04-99a2-208bed4d5010 None admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:59.268421 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2af82-92bc-4e04-99a2-208bed4d5010 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:59.268791 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2af82-92bc-4e04-99a2-208bed4d5010 None admin] PATH_INFO: `/v3/projects/cd1e6daca4264a0cbc11a9160d1213eb/users/7e42dfb7d22e4435889cb91327db5477/roles/4680aa64da80470b87075090fca2354b` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:59.285778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2af82-92bc-4e04-99a2-208bed4d5010 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd1e6daca4264a0cbc11a9160d1213eb, user_id=7e42dfb7d22e4435889cb91327db5477, role_id=4680aa64da80470b87075090fca2354b)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:59.286805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61dd0db-d834-4be2-80e7-43432bb6f4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:59.287267 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2af82-92bc-4e04-99a2-208bed4d5010 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:59.287456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dd0db-d834-4be2-80e7-43432bb6f4d1 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:59.287729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dd0db-d834-4be2-80e7-43432bb6f4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:59.288000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dd0db-d834-4be2-80e7-43432bb6f4d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:59.288661 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61dd0db-d834-4be2-80e7-43432bb6f4d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:59.288964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61dd0db-d834-4be2-80e7-43432bb6f4d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:59.291252 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61dd0db-d834-4be2-80e7-43432bb6f4d1 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:59.298227 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:59.304106 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 312/609] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 31 00:07:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:59.305259 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61dd0db-d834-4be2-80e7-43432bb6f4d1 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:59.308016 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 298/610] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 31 00:07:59 2022] PUT /identity/v3/projects/cd1e6daca4264a0cbc11a9160d1213eb/users/7e42dfb7d22e4435889cb91327db5477/roles/4680aa64da80470b87075090fca2354b => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:07:59.321003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2af82-92bc-4e04-99a2-208bed4d5010 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:59.427486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:59.463483 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61dd0db-d834-4be2-80e7-43432bb6f4d1 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:59.503464 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad605c-929d-4e25-944a-cfdf0c41ab51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:59.504632 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad605c-929d-4e25-944a-cfdf0c41ab51 None admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:59.504933 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad605c-929d-4e25-944a-cfdf0c41ab51 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:59.505213 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad605c-929d-4e25-944a-cfdf0c41ab51 None admin] PATH_INFO: `/v3/projects/cd1e6daca4264a0cbc11a9160d1213eb/users/05c0d9554cf94c41a2cd709cab378e53/roles/4680aa64da80470b87075090fca2354b` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:59.523302 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad605c-929d-4e25-944a-cfdf0c41ab51 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd1e6daca4264a0cbc11a9160d1213eb, user_id=05c0d9554cf94c41a2cd709cab378e53, role_id=4680aa64da80470b87075090fca2354b)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:59.525253 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad605c-929d-4e25-944a-cfdf0c41ab51 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:59.543477 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 313/611] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 31 00:07:59 2022] PUT /identity/v3/projects/cd1e6daca4264a0cbc11a9160d1213eb/users/05c0d9554cf94c41a2cd709cab378e53/roles/4680aa64da80470b87075090fca2354b => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:07:59.545942 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2af82-92bc-4e04-99a2-208bed4d5010 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:59.594229 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038860d-f907-4d71-87b2-c05c9981416b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:59.595245 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038860d-f907-4d71-87b2-c05c9981416b None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:59.595704 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038860d-f907-4d71-87b2-c05c9981416b None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:59.596149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038860d-f907-4d71-87b2-c05c9981416b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:59.609459 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d038860d-f907-4d71-87b2-c05c9981416b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:59.611263 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d038860d-f907-4d71-87b2-c05c9981416b None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:59.628471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-d038860d-f907-4d71-87b2-c05c9981416b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:59.628471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:07:59.628471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:07:59.628471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:07:59.628471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:07:59.628471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:07:59.628471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:07:59.628471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:07:59.628471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:07:59.628471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:07:59.628471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:07:59.628471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:07:59.628471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:07:59.628471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:07:59.628471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:07:59.628471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:07:59.629460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:07:59.629460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:07:59.629460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:07:59.629460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:07:59.629460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:07:59.629460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:07:59.629460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:07:59.629460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:07:59.629460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:07:59.629460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:07:59.629460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:07:59.629460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:07:59.629460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:07:59.629460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:07:59.629460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:07:59.629460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:07:59.629460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:07:59.630327 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:07:59.630327 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:07:59.630327 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:07:59.633005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 299/612] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:07:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:07:59.639706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad605c-929d-4e25-944a-cfdf0c41ab51 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:59.756834 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad605c-929d-4e25-944a-cfdf0c41ab51 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:07:59.775331 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:59.795123 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d115095-d345-4c12-acf7-53a821079c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:07:59.795954 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d115095-d345-4c12-acf7-53a821079c84 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:59.796406 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d115095-d345-4c12-acf7-53a821079c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:59.796952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d115095-d345-4c12-acf7-53a821079c84 None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:59.797602 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d115095-d345-4c12-acf7-53a821079c84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:07:59.797906 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d115095-d345-4c12-acf7-53a821079c84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:07:59.799325 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d115095-d345-4c12-acf7-53a821079c84 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:07:59.812076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 314/613] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:07:59.812781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f18985-a362-439a-bbde-934d7658b25a None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:59.813258 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f18985-a362-439a-bbde-934d7658b25a None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:59.813603 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f18985-a362-439a-bbde-934d7658b25a None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:59.815139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 300/614] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:07:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:07:59.817426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d115095-d345-4c12-acf7-53a821079c84 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:07:59.820059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcceb2b5-97ab-4801-b886-fc2829d71ae8 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:07:59.820348 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcceb2b5-97ab-4801-b886-fc2829d71ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:07:59.820625 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcceb2b5-97ab-4801-b886-fc2829d71ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:07:59.848483 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-bcceb2b5-97ab-4801-b886-fc2829d71ae8 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:07:59.849610 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-bcceb2b5-97ab-4801-b886-fc2829d71ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:07:59.935294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:07:59.964404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 301/615] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:07:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:07:59.994378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d115095-d345-4c12-acf7-53a821079c84 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:00.050690 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389df6a0-3685-4dce-84e0-8b882269d752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:00.051412 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389df6a0-3685-4dce-84e0-8b882269d752 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:00.051641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389df6a0-3685-4dce-84e0-8b882269d752 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:00.051912 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389df6a0-3685-4dce-84e0-8b882269d752 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:00.063500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389df6a0-3685-4dce-84e0-8b882269d752 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:00.065007 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389df6a0-3685-4dce-84e0-8b882269d752 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:00.082816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-389df6a0-3685-4dce-84e0-8b882269d752 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 00:08:00.082816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:00.082816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:00.082816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:00.082816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:00.082816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:00.082816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:00.082816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:00.082816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:00.082816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:00.082816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:00.082816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:00.082816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:08:00.082816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:08:00.082816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:08:00.082816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:08:00.083828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:00.083828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:00.083828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:08:00.083828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:00.083828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:08:00.083828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:00.083828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:00.083828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:00.083828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:00.083828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:00.083828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:00.083828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:00.083828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:00.083828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:00.083828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:00.083828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:00.083828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:00.083828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:00.084792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:00.084792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:00.084792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:08:00.084792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:08:00.084792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:08:00.084792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:08:00.084792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:08:00.084792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:08:00.084792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 00:08:00.084792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:00.087395 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 315/616] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:07:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:00.092221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcceb2b5-97ab-4801-b886-fc2829d71ae8 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:00.241986 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcceb2b5-97ab-4801-b886-fc2829d71ae8 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:00.286817 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119570ee-5375-4645-9b94-17f062a4fbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:00.287694 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119570ee-5375-4645-9b94-17f062a4fbf5 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:00.288118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119570ee-5375-4645-9b94-17f062a4fbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:00.288552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119570ee-5375-4645-9b94-17f062a4fbf5 None admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:00.289433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119570ee-5375-4645-9b94-17f062a4fbf5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:00.289879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119570ee-5375-4645-9b94-17f062a4fbf5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:00.291473 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119570ee-5375-4645-9b94-17f062a4fbf5 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:00.311008 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 302/617] 10.222.0.67 () {56 vars in 1135 bytes} [Wed Aug 31 00:08:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:00.319107 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389df6a0-3685-4dce-84e0-8b882269d752 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:00.354430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119570ee-5375-4645-9b94-17f062a4fbf5 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:00.398162 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:00.404073 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:00.470340 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119570ee-5375-4645-9b94-17f062a4fbf5 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:00.513160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c735c57-6f8b-4207-8558-378420ff3237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:00.514321 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c735c57-6f8b-4207-8558-378420ff3237 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:00.514764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c735c57-6f8b-4207-8558-378420ff3237 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:00.515219 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c735c57-6f8b-4207-8558-378420ff3237 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:00.526249 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c735c57-6f8b-4207-8558-378420ff3237 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:08:00.532002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-7c735c57-6f8b-4207-8558-378420ff3237 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:08:00.532002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:00.532002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:00.532002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:00.532002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:00.532002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:00.532002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:00.532002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:00.532002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:00.532002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:00.532002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:00.532002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:00.532002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:08:00.532002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:08:00.532002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:08:00.532002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:08:00.533111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:08:00.533111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:08:00.533111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:08:00.533111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:08:00.533111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:00.533111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:00.533111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:00.533111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:00.533111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:00.533111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:00.533111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:00.533111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:00.533111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:00.533111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:00.533111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:00.533111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:00.534108 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:00.534108 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:00.534108 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:00.534108 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:00.534108 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:08:00.534108 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:08:00.534108 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:08:00.534108 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:08:00.534108 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:08:00.534108 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:08:00.534108 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:08:00.534108 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:00.536518 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 303/618] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 31 00:08:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:00.542222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c735c57-6f8b-4207-8558-378420ff3237 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:00.563643 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389df6a0-3685-4dce-84e0-8b882269d752 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:00.624700 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f56b404-0bfb-41e5-926d-0893c1fae28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:00.625589 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56b404-0bfb-41e5-926d-0893c1fae28f None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:00.625868 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56b404-0bfb-41e5-926d-0893c1fae28f None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:00.626141 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56b404-0bfb-41e5-926d-0893c1fae28f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:00.638607 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f56b404-0bfb-41e5-926d-0893c1fae28f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:00.640124 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f56b404-0bfb-41e5-926d-0893c1fae28f None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:00.654039 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c735c57-6f8b-4207-8558-378420ff3237 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:00.656035 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:00.670489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-4f56b404-0bfb-41e5-926d-0893c1fae28f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:00.670489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:00.670489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:00.670489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:00.670489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:00.670489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:00.670489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:00.670489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:00.670489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:00.670489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:00.670489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:00.670489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:00.670489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:00.670489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:00.670489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:00.670489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:00.672111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:00.672111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:00.672111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:00.672111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:00.672111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:00.672111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:00.672111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:00.672111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:00.672111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:00.672111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:00.672111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:00.672111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:00.672111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:00.672111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:00.672111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:00.672111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:00.672111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:00.673044 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:00.673044 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:00.673044 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:00.676143 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 316/619] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:00.681576 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f56b404-0bfb-41e5-926d-0893c1fae28f None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:00.699806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59ca84-9765-476a-9714-61575a5d7620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:00.703461 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59ca84-9765-476a-9714-61575a5d7620 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:00.703993 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59ca84-9765-476a-9714-61575a5d7620 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:00.704466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59ca84-9765-476a-9714-61575a5d7620 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:00.705352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59ca84-9765-476a-9714-61575a5d7620 None admin] RBAC: Adding query filter params (name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:00.706091 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59ca84-9765-476a-9714-61575a5d7620 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:00.708776 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59ca84-9765-476a-9714-61575a5d7620 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:00.724569 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 304/620] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 31 00:08:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:00.740453 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59ca84-9765-476a-9714-61575a5d7620 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:00.832257 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:00.847266 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f56b404-0bfb-41e5-926d-0893c1fae28f None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:00.885120 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b42896-9dc4-4d98-bdf0-40a9965e1884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:00.886131 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b42896-9dc4-4d98-bdf0-40a9965e1884 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:00.886720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b42896-9dc4-4d98-bdf0-40a9965e1884 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:00.887233 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b42896-9dc4-4d98-bdf0-40a9965e1884 None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:00.890723 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b42896-9dc4-4d98-bdf0-40a9965e1884 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:00.891341 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b42896-9dc4-4d98-bdf0-40a9965e1884 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:00.892944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b42896-9dc4-4d98-bdf0-40a9965e1884 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:00.906857 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 317/621] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:00.912669 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b42896-9dc4-4d98-bdf0-40a9965e1884 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:00.983014 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59ca84-9765-476a-9714-61575a5d7620 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:01.027519 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f11e0-bd25-49ea-885c-3b9d4ca3478f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:01.028690 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f11e0-bd25-49ea-885c-3b9d4ca3478f None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:01.029123 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f11e0-bd25-49ea-885c-3b9d4ca3478f None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:01.029551 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f11e0-bd25-49ea-885c-3b9d4ca3478f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:01.041577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f11e0-bd25-49ea-885c-3b9d4ca3478f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:01.043230 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f11e0-bd25-49ea-885c-3b9d4ca3478f None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:01.063391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-eb2f11e0-bd25-49ea-885c-3b9d4ca3478f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:01.063391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:01.063391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:01.063391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:01.063391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:01.063391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:01.063391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:01.063391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:01.063391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:01.063391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:01.063391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:01.063391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:01.063391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:01.063391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:01.063391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:01.063391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:01.064538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:01.064538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:01.064538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:01.064538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:01.064538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:01.064538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:01.064538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:01.064538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:01.064538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:01.064538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:01.064538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:01.064538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:01.064538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:01.064538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:01.064538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:01.064538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:01.064538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:01.065615 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:01.065615 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:01.065615 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:01.067921 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 305/622] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:01.069281 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3404af9-b77a-4355-a14d-23e7686f5914 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:01.069704 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3404af9-b77a-4355-a14d-23e7686f5914 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:01.070099 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3404af9-b77a-4355-a14d-23e7686f5914 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:01.071727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 306/623] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:01.073292 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3404af9-b77a-4355-a14d-23e7686f5914 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:01.094287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:01.112803 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b42896-9dc4-4d98-bdf0-40a9965e1884 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:01.148961 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f7e38-3edf-4b26-896f-fd5a95e210ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:01.149765 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f7e38-3edf-4b26-896f-fd5a95e210ec None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:01.150168 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f7e38-3edf-4b26-896f-fd5a95e210ec None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:01.150552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f7e38-3edf-4b26-896f-fd5a95e210ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:01.158463 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f7e38-3edf-4b26-896f-fd5a95e210ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:01.160086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f7e38-3edf-4b26-896f-fd5a95e210ec None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:01.172384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-609f7e38-3edf-4b26-896f-fd5a95e210ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:01.172384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:01.172384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:01.172384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:01.172384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:01.172384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:01.172384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:01.172384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:01.172384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:01.172384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:01.172384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:01.172384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:01.172384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:08:01.172384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:08:01.172384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:08:01.173416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:08:01.173416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:01.173416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:01.173416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:01.173416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:01.173416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:01.173416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:01.173416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:01.173416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:01.173416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:01.173416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:01.173416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:01.173416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:01.173416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:01.173416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:01.173416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:01.173416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:01.174454 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:08:01.174454 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:08:01.174454 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:08:01.174454 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:08:01.174454 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:08:01.174454 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:08:01.174454 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:01.174454 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:01.175562 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 318/624] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:01.176859 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb984f7-bc07-4773-a6a6-3f822d1419fa None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:01.177249 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb984f7-bc07-4773-a6a6-3f822d1419fa None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:01.177630 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb984f7-bc07-4773-a6a6-3f822d1419fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:01.195971 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-bbb984f7-bc07-4773-a6a6-3f822d1419fa None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:01.197317 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-bbb984f7-bc07-4773-a6a6-3f822d1419fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:01.215248 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3404af9-b77a-4355-a14d-23e7686f5914 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:01.225004 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:01.267805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864b445-de96-4215-9bc0-0436a6dcadb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:01.269263 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864b445-de96-4215-9bc0-0436a6dcadb4 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:01.269709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864b445-de96-4215-9bc0-0436a6dcadb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:01.270329 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864b445-de96-4215-9bc0-0436a6dcadb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:01.271566 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864b445-de96-4215-9bc0-0436a6dcadb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:01.272286 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864b445-de96-4215-9bc0-0436a6dcadb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:01.273795 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864b445-de96-4215-9bc0-0436a6dcadb4 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:01.275640 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 319/625] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:08:01.284307 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb984f7-bc07-4773-a6a6-3f822d1419fa None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:01.289029 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 307/626] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:01.300707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864b445-de96-4215-9bc0-0436a6dcadb4 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:01.444161 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:01.464952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb984f7-bc07-4773-a6a6-3f822d1419fa None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:01.483627 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864b445-de96-4215-9bc0-0436a6dcadb4 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:01.523810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc33fa-6d7b-46b0-9d6f-8ae139a8e742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:01.524801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc33fa-6d7b-46b0-9d6f-8ae139a8e742 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:01.525284 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc33fa-6d7b-46b0-9d6f-8ae139a8e742 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:01.525788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc33fa-6d7b-46b0-9d6f-8ae139a8e742 None admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:01.526701 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc33fa-6d7b-46b0-9d6f-8ae139a8e742 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:01.527243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc33fa-6d7b-46b0-9d6f-8ae139a8e742 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:01.529592 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc33fa-6d7b-46b0-9d6f-8ae139a8e742 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:01.540790 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:01.549622 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 320/627] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:08:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:01.558293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc33fa-6d7b-46b0-9d6f-8ae139a8e742 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:01.559125 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff968d1-1c2e-46b5-b9d6-fb8b490b90a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:01.560183 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff968d1-1c2e-46b5-b9d6-fb8b490b90a9 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:01.560466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff968d1-1c2e-46b5-b9d6-fb8b490b90a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:01.560740 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff968d1-1c2e-46b5-b9d6-fb8b490b90a9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:01.568449 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff968d1-1c2e-46b5-b9d6-fb8b490b90a9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:01.569885 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff968d1-1c2e-46b5-b9d6-fb8b490b90a9 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:01.589571 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-4ff968d1-1c2e-46b5-b9d6-fb8b490b90a9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:08:01.589571 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:01.589571 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:01.589571 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:01.589571 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:01.589571 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:01.589571 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:01.589571 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:01.589571 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:01.589571 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:01.589571 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:01.589571 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:01.589571 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:08:01.589571 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:08:01.589571 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:08:01.589571 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:08:01.591127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:01.591127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:01.591127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:08:01.591127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:01.591127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:08:01.591127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:01.591127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:01.591127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:01.591127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:01.591127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:01.591127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:01.591127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:01.591127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:01.591127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:01.591127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:01.591127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:01.591127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:01.591127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:01.592159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:01.592159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:01.592159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:08:01.592159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:08:01.592159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:08:01.592159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:08:01.592159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:08:01.592159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:08:01.592159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:08:01.592159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:01.592159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 308/628] 10.222.0.67 () {56 vars in 1093 bytes} [Wed Aug 31 00:08:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:01.597186 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff968d1-1c2e-46b5-b9d6-fb8b490b90a9 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:01.722033 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff968d1-1c2e-46b5-b9d6-fb8b490b90a9 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:01.723291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc33fa-6d7b-46b0-9d6f-8ae139a8e742 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:01.757028 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:01.760107 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd23f5-07da-400f-8781-d65645720561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:01.760868 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd23f5-07da-400f-8781-d65645720561 None admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:01.761205 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd23f5-07da-400f-8781-d65645720561 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:01.761548 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd23f5-07da-400f-8781-d65645720561 None admin] PATH_INFO: `/v3/projects/cd1e6daca4264a0cbc11a9160d1213eb/users/1cd3cfbc8ebd4c5a83bef09efe10bc91/roles/4680aa64da80470b87075090fca2354b` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:01.763227 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368523c-dbaa-48f1-898b-b5d24156eb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:01.763858 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368523c-dbaa-48f1-898b-b5d24156eb91 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:01.764371 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368523c-dbaa-48f1-898b-b5d24156eb91 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:01.765100 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368523c-dbaa-48f1-898b-b5d24156eb91 None admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:01.766768 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368523c-dbaa-48f1-898b-b5d24156eb91 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:01.767173 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368523c-dbaa-48f1-898b-b5d24156eb91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:01.768600 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368523c-dbaa-48f1-898b-b5d24156eb91 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:01.777474 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd23f5-07da-400f-8781-d65645720561 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd1e6daca4264a0cbc11a9160d1213eb, user_id=1cd3cfbc8ebd4c5a83bef09efe10bc91, role_id=4680aa64da80470b87075090fca2354b)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:01.779894 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd23f5-07da-400f-8781-d65645720561 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:01.788229 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 309/629] 10.222.0.67 () {56 vars in 1133 bytes} [Wed Aug 31 00:08:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:01.795201 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 321/630] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 31 00:08:01 2022] PUT /identity/v3/projects/cd1e6daca4264a0cbc11a9160d1213eb/users/1cd3cfbc8ebd4c5a83bef09efe10bc91/roles/4680aa64da80470b87075090fca2354b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:08:01.795389 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368523c-dbaa-48f1-898b-b5d24156eb91 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:01.795482 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc6c5a-8921-4f84-9d1c-8b70554e9c5e None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:01.795482 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc6c5a-8921-4f84-9d1c-8b70554e9c5e None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:01.795482 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc6c5a-8921-4f84-9d1c-8b70554e9c5e None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:01.796146 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 322/631] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:01.797168 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc6c5a-8921-4f84-9d1c-8b70554e9c5e None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:01.863109 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:01.916634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368523c-dbaa-48f1-898b-b5d24156eb91 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:01.926269 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc6c5a-8921-4f84-9d1c-8b70554e9c5e None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:01.960392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad62476-be46-47bf-b5ae-f5c5d57f2fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:01.960944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad62476-be46-47bf-b5ae-f5c5d57f2fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:01.961213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad62476-be46-47bf-b5ae-f5c5d57f2fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:01.961472 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad62476-be46-47bf-b5ae-f5c5d57f2fb4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:01.969597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37419235-f615-40fc-9cae-f1ea9db1ae73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:01.970268 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37419235-f615-40fc-9cae-f1ea9db1ae73 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:01.970494 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37419235-f615-40fc-9cae-f1ea9db1ae73 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:01.970770 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37419235-f615-40fc-9cae-f1ea9db1ae73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:01.974515 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad62476-be46-47bf-b5ae-f5c5d57f2fb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:08:01.979765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-dad62476-be46-47bf-b5ae-f5c5d57f2fb4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:08:01.979765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:01.979765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:01.979765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:01.979765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:01.979765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:01.979765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:01.979765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:01.979765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:01.979765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:01.979765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:01.979765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:01.979765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:08:01.979765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:08:01.979765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:08:01.979765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:08:01.980718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:08:01.980718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:08:01.980718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:08:01.980718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:08:01.980718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:01.980718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:01.980718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:01.980718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:01.980718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:01.980718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:01.980718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:01.980718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:01.980718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:01.980718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:01.980718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:01.980718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:01.982007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:01.982007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:01.982007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:01.982007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:01.982007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:08:01.982007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:08:01.982007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:08:01.982007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:08:01.982007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:08:01.982007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:08:01.982007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:08:01.982007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:01.982007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 310/632] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 31 00:08:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:01.982981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbf29b-2da4-45b0-9cbd-5af9ebf374bb None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:01.982981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbf29b-2da4-45b0-9cbd-5af9ebf374bb None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:01.983159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbf29b-2da4-45b0-9cbd-5af9ebf374bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:02.002945 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-75dbf29b-2da4-45b0-9cbd-5af9ebf374bb None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:02.004350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-75dbf29b-2da4-45b0-9cbd-5af9ebf374bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:02.050662 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37419235-f615-40fc-9cae-f1ea9db1ae73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:02.052156 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37419235-f615-40fc-9cae-f1ea9db1ae73 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:02.057206 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:02.087598 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 311/633] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:08:02.089010 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acf6da-2705-467c-86e6-fbbbe2e3135f None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:02.089373 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acf6da-2705-467c-86e6-fbbbe2e3135f None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:02.089719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acf6da-2705-467c-86e6-fbbbe2e3135f None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:02.095916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 312/634] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:02.095916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93acf6da-2705-467c-86e6-fbbbe2e3135f None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:02.109233 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:02.184836 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 323/635] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:08:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:08:02.186628 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de35f9-f00e-41fa-bb98-03f3f53dcbc7 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:02.187018 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de35f9-f00e-41fa-bb98-03f3f53dcbc7 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:02.187409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de35f9-f00e-41fa-bb98-03f3f53dcbc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:02.206946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-a8de35f9-f00e-41fa-bb98-03f3f53dcbc7 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:02.208403 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a8de35f9-f00e-41fa-bb98-03f3f53dcbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:02.224934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93acf6da-2705-467c-86e6-fbbbe2e3135f None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:02.270481 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f3c40-98d0-4e2a-97f1-9212137851a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:02.272239 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f3c40-98d0-4e2a-97f1-9212137851a1 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:02.272779 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f3c40-98d0-4e2a-97f1-9212137851a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:02.273210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f3c40-98d0-4e2a-97f1-9212137851a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:02.274190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438f3c40-98d0-4e2a-97f1-9212137851a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:02.274970 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438f3c40-98d0-4e2a-97f1-9212137851a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:02.276967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438f3c40-98d0-4e2a-97f1-9212137851a1 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:02.291303 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 324/636] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:08:02.291303 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de35f9-f00e-41fa-bb98-03f3f53dcbc7 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:02.291675 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 313/637] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 31 00:08:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:02.299220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f3c40-98d0-4e2a-97f1-9212137851a1 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:02.343626 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:02.408059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:02.410183 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f3c40-98d0-4e2a-97f1-9212137851a1 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:02.433619 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de35f9-f00e-41fa-bb98-03f3f53dcbc7 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:02.450851 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017aba0a-2588-43fe-a51a-8cac58a99645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:02.451914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017aba0a-2588-43fe-a51a-8cac58a99645 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:02.452516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017aba0a-2588-43fe-a51a-8cac58a99645 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:02.452966 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017aba0a-2588-43fe-a51a-8cac58a99645 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:02.467811 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017aba0a-2588-43fe-a51a-8cac58a99645 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:02.469279 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017aba0a-2588-43fe-a51a-8cac58a99645 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:02.475535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede2ef2-f281-403e-b562-6b7fed689798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:02.476116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede2ef2-f281-403e-b562-6b7fed689798 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:02.476450 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede2ef2-f281-403e-b562-6b7fed689798 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:02.476729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede2ef2-f281-403e-b562-6b7fed689798 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:02.484775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-017aba0a-2588-43fe-a51a-8cac58a99645 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:02.484775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:02.484775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:02.484775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:02.484775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:02.484775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:02.484775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:02.484775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:02.484775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:02.484775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:02.484775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:02.484775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:02.484775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:02.484775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:02.484775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:02.484775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:02.485748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:02.485748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:02.485748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:02.485748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:02.485748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:02.485748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:02.485748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:02.485748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:02.485748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:02.485748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:02.485748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:02.485748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:02.485748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:02.485748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:02.485748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:02.485748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:02.485748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:02.486631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:02.486631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:02.486631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:02.486631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 314/638] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:02.487299 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017aba0a-2588-43fe-a51a-8cac58a99645 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:02.495056 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede2ef2-f281-403e-b562-6b7fed689798 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:02.495056 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede2ef2-f281-403e-b562-6b7fed689798 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:02.518545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-9ede2ef2-f281-403e-b562-6b7fed689798 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:02.518545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:02.518545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:02.518545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:02.518545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:02.518545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:02.518545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:02.518545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:02.518545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:02.518545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:02.518545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:02.518545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:02.518545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:02.518545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:02.518545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:02.518545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:02.521747 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:02.521747 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:02.521747 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:02.521747 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:02.521747 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:02.521747 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:02.521747 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:02.521747 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:02.521747 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:02.521747 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:02.521747 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:02.521747 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:02.521747 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:02.521747 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:02.521747 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:02.521747 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:02.521747 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:02.524668 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:02.524668 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:02.524668 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:02.524668 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 325/639] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:02.524668 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede2ef2-f281-403e-b562-6b7fed689798 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:02.614784 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:02.623377 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017aba0a-2588-43fe-a51a-8cac58a99645 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:02.659873 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd87301-2610-409a-9161-4a0b48afc42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:02.660558 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd87301-2610-409a-9161-4a0b48afc42b None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:02.660826 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd87301-2610-409a-9161-4a0b48afc42b None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:02.661088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd87301-2610-409a-9161-4a0b48afc42b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:02.666858 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede2ef2-f281-403e-b562-6b7fed689798 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:02.671513 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd87301-2610-409a-9161-4a0b48afc42b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:08:02.676725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-2cd87301-2610-409a-9161-4a0b48afc42b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:08:02.676725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:02.676725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:02.676725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:02.676725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:02.676725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:02.676725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:02.676725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:02.676725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:02.676725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:02.676725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:02.676725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:02.676725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:08:02.676725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:08:02.676725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:08:02.676725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:08:02.679234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:08:02.679234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:08:02.679234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:08:02.679234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:08:02.679234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:02.679234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:02.679234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:02.679234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:02.679234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:02.679234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:02.679234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:02.679234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:02.679234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:02.679234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:02.679234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:02.679234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:02.680435 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:02.680435 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:02.680435 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:02.680435 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:02.680435 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:08:02.680435 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:08:02.680435 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:08:02.680435 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:08:02.680435 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:08:02.680435 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:08:02.680435 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:08:02.680435 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:02.680435 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 315/640] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 31 00:08:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:02.680435 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd87301-2610-409a-9161-4a0b48afc42b None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:02.708086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:02.717544 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d9290-4d21-4382-99d1-d77f7b7bda6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:02.718343 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d9290-4d21-4382-99d1-d77f7b7bda6a None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:02.719424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d9290-4d21-4382-99d1-d77f7b7bda6a None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:02.719424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d9290-4d21-4382-99d1-d77f7b7bda6a None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:02.720390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d9290-4d21-4382-99d1-d77f7b7bda6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:02.720866 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d9290-4d21-4382-99d1-d77f7b7bda6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:02.722680 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d9290-4d21-4382-99d1-d77f7b7bda6a None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:02.739360 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 326/641] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:02.740655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d9290-4d21-4382-99d1-d77f7b7bda6a None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:02.790661 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd87301-2610-409a-9161-4a0b48afc42b None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:02.827329 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50a3b2-cf6e-4435-a786-df1d4edf521e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:02.828287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50a3b2-cf6e-4435-a786-df1d4edf521e None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:02.828570 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50a3b2-cf6e-4435-a786-df1d4edf521e None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:02.828841 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50a3b2-cf6e-4435-a786-df1d4edf521e None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:02.829483 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50a3b2-cf6e-4435-a786-df1d4edf521e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:02.829788 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50a3b2-cf6e-4435-a786-df1d4edf521e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:02.831181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50a3b2-cf6e-4435-a786-df1d4edf521e None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:02.848735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 316/642] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:02.848735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50a3b2-cf6e-4435-a786-df1d4edf521e None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:02.863704 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:02.875143 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d9290-4d21-4382-99d1-d77f7b7bda6a None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:02.921150 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5236e69-8600-42bf-a65c-6ec19604a852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:02.922041 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5236e69-8600-42bf-a65c-6ec19604a852 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:02.922477 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5236e69-8600-42bf-a65c-6ec19604a852 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:02.922900 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5236e69-8600-42bf-a65c-6ec19604a852 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:02.937557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5236e69-8600-42bf-a65c-6ec19604a852 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:08:02.945776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-a5236e69-8600-42bf-a65c-6ec19604a852 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:08:02.945776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:02.945776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:02.945776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:02.945776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:02.945776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:02.945776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:02.945776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:02.945776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:02.945776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:02.945776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:02.945776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:02.945776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:08:02.945776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:08:02.945776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:08:02.945776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:08:02.946965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:08:02.946965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:08:02.946965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:08:02.946965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:08:02.946965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:02.946965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:02.946965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:02.946965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:02.946965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:02.946965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:02.946965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:02.946965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:02.946965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:02.946965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:02.946965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:02.946965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:02.947852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:02.947852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:02.947852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:02.947852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:02.947852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:08:02.947852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:08:02.947852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:08:02.947852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:08:02.947852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:08:02.947852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:08:02.947852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:08:02.947852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:02.947852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 327/643] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 31 00:08:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:02.948969 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5236e69-8600-42bf-a65c-6ec19604a852 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:02.965316 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50a3b2-cf6e-4435-a786-df1d4edf521e None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:03.005422 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25326a93-ece1-42d2-b669-f22e98ac3f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:03.006032 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25326a93-ece1-42d2-b669-f22e98ac3f5b None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:03.006312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25326a93-ece1-42d2-b669-f22e98ac3f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:03.006581 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25326a93-ece1-42d2-b669-f22e98ac3f5b None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:03.007265 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25326a93-ece1-42d2-b669-f22e98ac3f5b None admin] RBAC: Adding query filter params (name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:03.007568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25326a93-ece1-42d2-b669-f22e98ac3f5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:03.008927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25326a93-ece1-42d2-b669-f22e98ac3f5b None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:03.018148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 317/644] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 31 00:08:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:03.018299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:03.019070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25326a93-ece1-42d2-b669-f22e98ac3f5b None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:03.089140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5236e69-8600-42bf-a65c-6ec19604a852 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:03.095861 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:03.120044 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25326a93-ece1-42d2-b669-f22e98ac3f5b None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:03.132196 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590ade39-7dd6-484a-9983-ff3d21941525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:03.133073 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590ade39-7dd6-484a-9983-ff3d21941525 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:03.133595 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590ade39-7dd6-484a-9983-ff3d21941525 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:03.134026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590ade39-7dd6-484a-9983-ff3d21941525 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:03.144133 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590ade39-7dd6-484a-9983-ff3d21941525 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:03.146180 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590ade39-7dd6-484a-9983-ff3d21941525 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:03.155400 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bb1c3-7951-4fcc-a451-898eb41b360e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:03.156024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bb1c3-7951-4fcc-a451-898eb41b360e None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:03.156302 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bb1c3-7951-4fcc-a451-898eb41b360e None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:03.156572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bb1c3-7951-4fcc-a451-898eb41b360e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:03.163213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29bb1c3-7951-4fcc-a451-898eb41b360e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:03.164656 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29bb1c3-7951-4fcc-a451-898eb41b360e None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:03.165481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-590ade39-7dd6-484a-9983-ff3d21941525 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 00:08:03.165481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:03.165481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:03.165481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:03.165481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:03.165481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:03.165481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:03.165481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:03.165481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:03.165481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:03.165481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:03.165481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:03.165481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:08:03.165481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:08:03.165481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:08:03.165481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:08:03.166361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:03.166361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:03.166361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:08:03.166361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:03.166361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:08:03.166361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:03.166361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:03.166361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:03.166361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:03.166361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:03.166361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:03.166361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:03.166361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:03.166361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:03.166361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:03.166361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:03.166361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:03.166361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:03.167456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:03.167456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:03.167456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:08:03.167456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:08:03.167456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:08:03.167456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:08:03.167456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:08:03.167456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:08:03.167456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 00:08:03.167456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:03.167456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 328/645] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:08:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:03.168070 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590ade39-7dd6-484a-9983-ff3d21941525 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:03.181364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-d29bb1c3-7951-4fcc-a451-898eb41b360e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:03.181364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:03.181364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:03.181364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:03.181364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:03.181364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:03.181364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:03.181364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:03.181364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:03.181364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:03.181364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:03.181364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:03.181364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:08:03.181364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:08:03.181364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:08:03.182282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:08:03.182282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:03.182282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:03.182282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:03.182282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:03.182282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:03.182282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:03.182282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:03.182282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:03.182282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:03.182282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:03.182282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:03.182282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:03.182282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:03.182282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:03.182282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:03.182282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:03.183817 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:08:03.183817 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:08:03.183817 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:08:03.183817 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:08:03.183817 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:08:03.183817 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:08:03.183817 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:03.183817 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:03.183817 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 318/646] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:03.183817 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bb1c3-7951-4fcc-a451-898eb41b360e None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:03.292790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bb1c3-7951-4fcc-a451-898eb41b360e None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:03.294829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590ade39-7dd6-484a-9983-ff3d21941525 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:03.306953 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:03.333644 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a62f0-ae80-4c12-83ff-5f932d5147bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:03.333884 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffb446-d77a-4fc0-b322-d7df1b74b7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:03.334280 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a62f0-ae80-4c12-83ff-5f932d5147bc None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:03.334397 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffb446-d77a-4fc0-b322-d7df1b74b7a0 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:03.334563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a62f0-ae80-4c12-83ff-5f932d5147bc None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:03.334671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffb446-d77a-4fc0-b322-d7df1b74b7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:03.334820 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a62f0-ae80-4c12-83ff-5f932d5147bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:03.334921 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffb446-d77a-4fc0-b322-d7df1b74b7a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:03.335563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffb446-d77a-4fc0-b322-d7df1b74b7a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:03.335864 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffb446-d77a-4fc0-b322-d7df1b74b7a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:03.337222 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffb446-d77a-4fc0-b322-d7df1b74b7a0 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:03.338009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:03.351095 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 329/647] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 31 00:08:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:03.351095 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffb446-d77a-4fc0-b322-d7df1b74b7a0 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:03.352909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a62f0-ae80-4c12-83ff-5f932d5147bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:03.355449 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a62f0-ae80-4c12-83ff-5f932d5147bc None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:03.370542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-815a62f0-ae80-4c12-83ff-5f932d5147bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:03.370542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:03.370542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:03.370542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:03.370542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:03.370542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:03.370542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:03.370542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:03.370542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:03.370542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:03.370542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:03.370542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:03.370542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:03.370542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:03.370542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:03.370542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:03.371719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:03.371719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:03.371719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:03.371719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:03.371719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:03.371719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:03.371719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:03.371719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:03.371719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:03.371719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:03.371719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:03.371719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:03.371719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:03.371719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:03.371719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:03.371719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:03.371719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:03.373632 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:03.373632 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:03.373632 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:03.377587 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 319/648] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:03.378396 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a62f0-ae80-4c12-83ff-5f932d5147bc None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:03.457571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffb446-d77a-4fc0-b322-d7df1b74b7a0 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:03.491763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a62f0-ae80-4c12-83ff-5f932d5147bc None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:03.493997 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faea8c7-027d-491a-8bbf-0992cd1ed091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:03.494605 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faea8c7-027d-491a-8bbf-0992cd1ed091 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:03.494883 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faea8c7-027d-491a-8bbf-0992cd1ed091 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:03.495183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faea8c7-027d-491a-8bbf-0992cd1ed091 None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:03.495907 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faea8c7-027d-491a-8bbf-0992cd1ed091 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:03.496210 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faea8c7-027d-491a-8bbf-0992cd1ed091 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:03.498259 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faea8c7-027d-491a-8bbf-0992cd1ed091 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:03.521175 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 330/649] 10.222.0.67 () {56 vars in 1137 bytes} [Wed Aug 31 00:08:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:03.521175 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faea8c7-027d-491a-8bbf-0992cd1ed091 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:03.531796 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e41c8-b752-481a-9092-77fc2d01e294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:03.532394 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e41c8-b752-481a-9092-77fc2d01e294 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:03.532670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e41c8-b752-481a-9092-77fc2d01e294 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:03.532936 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e41c8-b752-481a-9092-77fc2d01e294 None admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:03.533623 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e41c8-b752-481a-9092-77fc2d01e294 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:03.533922 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e41c8-b752-481a-9092-77fc2d01e294 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:03.535366 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e41c8-b752-481a-9092-77fc2d01e294 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:03.547587 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 320/650] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:08:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:03.548404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e41c8-b752-481a-9092-77fc2d01e294 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:03.561923 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:03.599690 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:03.636473 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faea8c7-027d-491a-8bbf-0992cd1ed091 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:03.656607 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e41c8-b752-481a-9092-77fc2d01e294 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:03.675708 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d361d-57bc-40be-b394-82747a196560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:03.676661 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d361d-57bc-40be-b394-82747a196560 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:03.677584 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d361d-57bc-40be-b394-82747a196560 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:03.678067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d361d-57bc-40be-b394-82747a196560 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:03.691991 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d361d-57bc-40be-b394-82747a196560 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:03.692692 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5a463-7227-47c3-8b3e-6c444dcf43ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:03.693306 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5a463-7227-47c3-8b3e-6c444dcf43ee None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:03.693489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d361d-57bc-40be-b394-82747a196560 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:03.693586 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5a463-7227-47c3-8b3e-6c444dcf43ee None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:03.693842 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5a463-7227-47c3-8b3e-6c444dcf43ee None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:03.694478 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5a463-7227-47c3-8b3e-6c444dcf43ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:03.694779 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5a463-7227-47c3-8b3e-6c444dcf43ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:03.696844 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5a463-7227-47c3-8b3e-6c444dcf43ee None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:03.709232 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 321/651] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:03.709604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-ec7d361d-57bc-40be-b394-82747a196560 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:03.709604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:03.709604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:03.709604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:03.709604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:03.709604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:03.709604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:03.709604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:03.709604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:03.709604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:03.709604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:03.709604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:03.709604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:03.709604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:03.709604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:03.709604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:03.710486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:03.710486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:03.710486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:03.710486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:03.710486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:03.710486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:03.710486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:03.710486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:03.710486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:03.710486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:03.710486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:03.710486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:03.710486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:03.710486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:03.710486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:03.710486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:03.710486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:03.711418 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:03.711418 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:03.711418 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:03.712758 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5a463-7227-47c3-8b3e-6c444dcf43ee None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:03.716378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 331/652] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:03.718174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d361d-57bc-40be-b394-82747a196560 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:03.804820 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:03.819113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5a463-7227-47c3-8b3e-6c444dcf43ee None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:03.823900 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d361d-57bc-40be-b394-82747a196560 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:03.828005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:03.853466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38dbb85-4fa7-498c-9357-09a1541d3067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:03.854075 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38dbb85-4fa7-498c-9357-09a1541d3067 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:03.854352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38dbb85-4fa7-498c-9357-09a1541d3067 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:03.854619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38dbb85-4fa7-498c-9357-09a1541d3067 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:03.859306 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fcfc9-eb01-4fca-941a-7f023aa63ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:03.859936 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fcfc9-eb01-4fca-941a-7f023aa63ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:03.860215 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fcfc9-eb01-4fca-941a-7f023aa63ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:03.860487 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fcfc9-eb01-4fca-941a-7f023aa63ba9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:03.861211 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3fcfc9-eb01-4fca-941a-7f023aa63ba9 None admin] RBAC: Adding query filter params (role.id=4680aa64da80470b87075090fca2354b, scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb, user.id=3aa53ae5a30146f69d7d6eefe4290dba) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:03.861522 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3fcfc9-eb01-4fca-941a-7f023aa63ba9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:03.862948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3fcfc9-eb01-4fca-941a-7f023aa63ba9 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:03.869011 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 332/653] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:08:03 2022] GET /identity/v3/role_assignments?user.id=3aa53ae5a30146f69d7d6eefe4290dba&scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb&role.id=4680aa64da80470b87075090fca2354b => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:03.869796 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fcfc9-eb01-4fca-941a-7f023aa63ba9 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:03.944705 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38dbb85-4fa7-498c-9357-09a1541d3067 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:03.946301 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38dbb85-4fa7-498c-9357-09a1541d3067 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:03.998399 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fcfc9-eb01-4fca-941a-7f023aa63ba9 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:04.041055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed18f8-1d13-4be1-bee5-306549e0ae2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:04.042382 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed18f8-1d13-4be1-bee5-306549e0ae2a None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:04.042382 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed18f8-1d13-4be1-bee5-306549e0ae2a None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:04.042382 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed18f8-1d13-4be1-bee5-306549e0ae2a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:04.053132 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed18f8-1d13-4be1-bee5-306549e0ae2a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:04.054598 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed18f8-1d13-4be1-bee5-306549e0ae2a None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:04.065106 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 322/654] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:08:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:08:04.065106 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38dbb85-4fa7-498c-9357-09a1541d3067 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:04.066629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-18ed18f8-1d13-4be1-bee5-306549e0ae2a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 00:08:04.066629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:04.066629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:04.066629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:04.066629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:04.066629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:04.066629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:04.066629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:04.066629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:04.066629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:04.066629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:04.066629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:04.066629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:08:04.066629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:08:04.066629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:08:04.066629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:08:04.067567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:04.067567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:04.067567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:08:04.067567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:04.067567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:08:04.067567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:04.067567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:04.067567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:04.067567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:04.067567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:04.067567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:04.067567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:04.067567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:04.067567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:04.067567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:04.067567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:04.067567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:04.067567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:04.068511 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:04.068511 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:04.068511 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:08:04.068511 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:08:04.068511 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:08:04.068511 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:08:04.068511 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:08:04.068511 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:08:04.068511 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 00:08:04.068511 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:04.068511 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 333/655] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:08:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:04.069582 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589af1c0-9399-4f09-870f-bce12d3148bd None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:04.069679 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589af1c0-9399-4f09-870f-bce12d3148bd None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:04.070558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589af1c0-9399-4f09-870f-bce12d3148bd None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:04.071252 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 334/656] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:04.075983 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589af1c0-9399-4f09-870f-bce12d3148bd None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:04.084065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:04.086186 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:04.188066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589af1c0-9399-4f09-870f-bce12d3148bd None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:04.188494 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38dbb85-4fa7-498c-9357-09a1541d3067 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:04.221910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d4c4d-d606-4810-b4c2-4212a2bf5d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:04.222545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d4c4d-d606-4810-b4c2-4212a2bf5d2f None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:04.222812 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d4c4d-d606-4810-b4c2-4212a2bf5d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:04.223115 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d4c4d-d606-4810-b4c2-4212a2bf5d2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:04.231866 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ecd691-ffbb-4776-aaf6-2e11a1585a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:04.233199 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ecd691-ffbb-4776-aaf6-2e11a1585a7f None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:04.233806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ecd691-ffbb-4776-aaf6-2e11a1585a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:04.234368 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ecd691-ffbb-4776-aaf6-2e11a1585a7f None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:04.234772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d4c4d-d606-4810-b4c2-4212a2bf5d2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:04.235431 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ecd691-ffbb-4776-aaf6-2e11a1585a7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:04.235916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ecd691-ffbb-4776-aaf6-2e11a1585a7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:04.236284 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d4c4d-d606-4810-b4c2-4212a2bf5d2f None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:04.237883 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ecd691-ffbb-4776-aaf6-2e11a1585a7f None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:04.260457 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 323/657] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:04.260643 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-a96d4c4d-d606-4810-b4c2-4212a2bf5d2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:04.260643 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:04.260643 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:04.260643 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:04.260643 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:04.260643 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:04.260643 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:04.260643 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:04.260643 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:04.260643 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:04.260643 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:04.260643 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:04.260643 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:04.260643 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:04.260643 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:04.260643 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:04.261546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:04.261546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:04.261546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:04.261546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:04.261546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:04.261546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:04.261546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:04.261546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:04.261546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:04.261546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:04.261546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:04.261546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:04.261546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:04.261546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:04.261546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:04.261546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:04.261546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:04.263069 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:04.263069 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:04.263069 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:04.263251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ecd691-ffbb-4776-aaf6-2e11a1585a7f None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:04.263340 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 335/658] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:04.263340 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38214362-6432-468a-9686-094cbbd15ef1 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:04.263340 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38214362-6432-468a-9686-094cbbd15ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:04.263340 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38214362-6432-468a-9686-094cbbd15ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:04.279833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-38214362-6432-468a-9686-094cbbd15ef1 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:04.281145 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-38214362-6432-468a-9686-094cbbd15ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:04.333797 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:04.341872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:04.357921 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 336/659] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:08:04.358844 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38214362-6432-468a-9686-094cbbd15ef1 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:04.381449 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ecd691-ffbb-4776-aaf6-2e11a1585a7f None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:04.428548 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a87da-1ce8-48d7-ae0e-1ffc48069557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:04.429739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a87da-1ce8-48d7-ae0e-1ffc48069557 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:04.430802 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a87da-1ce8-48d7-ae0e-1ffc48069557 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:04.431345 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a87da-1ce8-48d7-ae0e-1ffc48069557 None admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:04.432579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a87da-1ce8-48d7-ae0e-1ffc48069557 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:04.433092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a87da-1ce8-48d7-ae0e-1ffc48069557 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:04.437936 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a87da-1ce8-48d7-ae0e-1ffc48069557 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:04.458013 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 324/660] 10.222.0.67 () {56 vars in 1137 bytes} [Wed Aug 31 00:08:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:04.458976 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a87da-1ce8-48d7-ae0e-1ffc48069557 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:04.483254 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38214362-6432-468a-9686-094cbbd15ef1 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:04.533270 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f685ab6-a3f1-4d91-b117-afa033f92590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:04.534249 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f685ab6-a3f1-4d91-b117-afa033f92590 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:04.534953 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f685ab6-a3f1-4d91-b117-afa033f92590 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:04.535450 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f685ab6-a3f1-4d91-b117-afa033f92590 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:04.545540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f685ab6-a3f1-4d91-b117-afa033f92590 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:04.548597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f685ab6-a3f1-4d91-b117-afa033f92590 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:04.567197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-0f685ab6-a3f1-4d91-b117-afa033f92590 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 00:08:04.567197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:04.567197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:04.567197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:04.567197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:04.567197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:04.567197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:04.567197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:04.567197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:04.567197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:04.567197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:04.567197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:04.567197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:08:04.567197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:08:04.567197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:08:04.567197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:08:04.568171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:04.568171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:04.568171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:08:04.568171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:04.568171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:08:04.568171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:04.568171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:04.568171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:04.568171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:04.568171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:04.568171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:04.568171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:04.568171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:04.568171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:04.568171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:04.568171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:04.568171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:04.568171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:04.569094 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:04.569094 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:04.569094 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:08:04.569094 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:08:04.569094 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:08:04.569094 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:08:04.569094 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:08:04.569094 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:08:04.569094 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 00:08:04.569094 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:04.571493 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 337/661] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 31 00:08:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:04.573106 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f685ab6-a3f1-4d91-b117-afa033f92590 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:04.589484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a87da-1ce8-48d7-ae0e-1ffc48069557 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:04.626140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:04.631644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:04.639400 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac384919-2b17-4719-ba33-4f365ca15c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:04.639993 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac384919-2b17-4719-ba33-4f365ca15c6a None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:04.640269 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac384919-2b17-4719-ba33-4f365ca15c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:04.640538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac384919-2b17-4719-ba33-4f365ca15c6a None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:04.641517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac384919-2b17-4719-ba33-4f365ca15c6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:04.641802 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac384919-2b17-4719-ba33-4f365ca15c6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:04.643199 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac384919-2b17-4719-ba33-4f365ca15c6a None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:04.654876 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 325/662] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:04.655540 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac384919-2b17-4719-ba33-4f365ca15c6a None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:04.695358 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f685ab6-a3f1-4d91-b117-afa033f92590 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:04.734161 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8718f-1165-4a9e-a4fc-b7235c80bbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:04.734910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8718f-1165-4a9e-a4fc-b7235c80bbfd None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:04.735117 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8718f-1165-4a9e-a4fc-b7235c80bbfd None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:04.735323 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8718f-1165-4a9e-a4fc-b7235c80bbfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:04.761735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac384919-2b17-4719-ba33-4f365ca15c6a None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:04.799232 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ca506-4683-42a5-828c-1dd071d3f73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:04.800096 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ca506-4683-42a5-828c-1dd071d3f73e None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:04.800547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ca506-4683-42a5-828c-1dd071d3f73e None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:04.800825 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ca506-4683-42a5-828c-1dd071d3f73e None admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:04.801555 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4ca506-4683-42a5-828c-1dd071d3f73e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:04.801889 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4ca506-4683-42a5-828c-1dd071d3f73e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:04.803336 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4ca506-4683-42a5-828c-1dd071d3f73e None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:04.807140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8718f-1165-4a9e-a4fc-b7235c80bbfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:04.808729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8718f-1165-4a9e-a4fc-b7235c80bbfd None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:04.828260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 326/663] 10.222.0.67 () {56 vars in 1139 bytes} [Wed Aug 31 00:08:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:04.828260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ca506-4683-42a5-828c-1dd071d3f73e None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:04.862459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:04.881771 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:04.920452 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 338/664] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:08:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:08:04.921301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8718f-1165-4a9e-a4fc-b7235c80bbfd None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:04.945961 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ca506-4683-42a5-828c-1dd071d3f73e None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:04.983336 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56d04d-658f-4030-9ef2-04a2f5f39472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:04.983968 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56d04d-658f-4030-9ef2-04a2f5f39472 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:04.984245 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56d04d-658f-4030-9ef2-04a2f5f39472 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:04.984513 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56d04d-658f-4030-9ef2-04a2f5f39472 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:04.995102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb56d04d-658f-4030-9ef2-04a2f5f39472 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:08:05.000996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-cb56d04d-658f-4030-9ef2-04a2f5f39472 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:08:05.000996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:05.000996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:05.000996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:05.000996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:05.000996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:05.000996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:05.000996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:05.000996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:05.000996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:05.000996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:05.000996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:05.000996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:08:05.000996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:08:05.000996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:08:05.000996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:08:05.001934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:08:05.001934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:08:05.001934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:08:05.001934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:08:05.001934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:05.001934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:05.001934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:05.001934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:05.001934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:05.001934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:05.001934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:05.001934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:05.001934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:05.001934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:05.001934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:05.001934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:05.002765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:05.002765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:05.002765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:05.002765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:05.002765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:08:05.002765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:08:05.002765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:08:05.002765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:08:05.002765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:08:05.002765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:08:05.002765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:08:05.002765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:05.002765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 327/665] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 31 00:08:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:05.004176 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56d04d-658f-4030-9ef2-04a2f5f39472 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:05.041221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8718f-1165-4a9e-a4fc-b7235c80bbfd None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:05.079299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59cd20-ae46-48b4-8a56-c1513ed934dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:05.079957 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59cd20-ae46-48b4-8a56-c1513ed934dd None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:05.080238 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59cd20-ae46-48b4-8a56-c1513ed934dd None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:05.080510 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59cd20-ae46-48b4-8a56-c1513ed934dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:05.087759 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59cd20-ae46-48b4-8a56-c1513ed934dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:05.089208 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59cd20-ae46-48b4-8a56-c1513ed934dd None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:05.100780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-6b59cd20-ae46-48b4-8a56-c1513ed934dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:05.100780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:05.100780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:05.100780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:05.100780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:05.100780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:05.100780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:05.100780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:05.100780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:05.100780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:05.100780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:05.100780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:05.100780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:08:05.100780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:08:05.100780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:08:05.101641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:08:05.101641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:05.101641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:05.101641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:05.101641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:05.101641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:05.101641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:05.101641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:05.101641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:05.101641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:05.101641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:05.101641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:05.101641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:05.101641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:05.101641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:05.101641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:05.101641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:05.102546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:08:05.102546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:08:05.102546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:08:05.102546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:08:05.102546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:08:05.102546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:08:05.102546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:05.102546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:05.102546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 339/666] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:05.103056 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59cd20-ae46-48b4-8a56-c1513ed934dd None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:05.115244 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:05.117464 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56d04d-658f-4030-9ef2-04a2f5f39472 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:05.119749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:05.153854 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa5db19-741e-450a-a521-2b4efd7cb5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:05.154562 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5db19-741e-450a-a521-2b4efd7cb5a1 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:05.154825 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5db19-741e-450a-a521-2b4efd7cb5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:05.155122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5db19-741e-450a-a521-2b4efd7cb5a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:05.211210 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59cd20-ae46-48b4-8a56-c1513ed934dd None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:05.224254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa5db19-741e-450a-a521-2b4efd7cb5a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:05.225702 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa5db19-741e-450a-a521-2b4efd7cb5a1 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:05.247864 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3384bc-3147-4b57-aa09-c6dd9e71f895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:05.248663 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3384bc-3147-4b57-aa09-c6dd9e71f895 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:05.249002 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3384bc-3147-4b57-aa09-c6dd9e71f895 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:05.249326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3384bc-3147-4b57-aa09-c6dd9e71f895 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:05.260105 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3384bc-3147-4b57-aa09-c6dd9e71f895 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:05.261544 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3384bc-3147-4b57-aa09-c6dd9e71f895 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:05.278594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-8b3384bc-3147-4b57-aa09-c6dd9e71f895 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:05.278594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:05.278594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:05.278594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:05.278594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:05.278594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:05.278594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:05.278594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:05.278594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:05.278594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:05.278594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:05.278594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:05.278594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:05.278594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:05.278594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:05.278594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:05.279564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:05.279564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:05.279564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:05.279564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:05.279564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:05.279564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:05.279564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:05.279564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:05.279564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:05.279564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:05.279564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:05.279564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:05.279564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:05.279564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:05.279564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:05.279564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:05.279564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:05.280443 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:05.280443 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:05.280443 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:05.280443 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 340/667] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:05.280688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3384bc-3147-4b57-aa09-c6dd9e71f895 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:05.336734 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 328/668] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:08:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:08:05.342690 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa5db19-741e-450a-a521-2b4efd7cb5a1 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:05.348170 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:05.355914 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:05.411584 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3384bc-3147-4b57-aa09-c6dd9e71f895 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:05.448386 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa5db19-741e-450a-a521-2b4efd7cb5a1 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:05.452841 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23065c6f-cd37-4285-a864-7208b6c83ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:05.453637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23065c6f-cd37-4285-a864-7208b6c83ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:05.454093 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23065c6f-cd37-4285-a864-7208b6c83ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:05.454523 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23065c6f-cd37-4285-a864-7208b6c83ef6 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:05.455372 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23065c6f-cd37-4285-a864-7208b6c83ef6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:05.455949 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23065c6f-cd37-4285-a864-7208b6c83ef6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:05.457328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23065c6f-cd37-4285-a864-7208b6c83ef6 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:05.471637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 341/669] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 31 00:08:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:05.471637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23065c6f-cd37-4285-a864-7208b6c83ef6 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:05.487136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20415f44-43c4-4692-9781-239af89d1e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:05.487754 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20415f44-43c4-4692-9781-239af89d1e82 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:05.488032 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20415f44-43c4-4692-9781-239af89d1e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:05.488296 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20415f44-43c4-4692-9781-239af89d1e82 None admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:05.488988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20415f44-43c4-4692-9781-239af89d1e82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:05.489293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20415f44-43c4-4692-9781-239af89d1e82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:05.490707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20415f44-43c4-4692-9781-239af89d1e82 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:05.508252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 329/670] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:08:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:05.508252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20415f44-43c4-4692-9781-239af89d1e82 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:05.569682 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:05.587810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23065c6f-cd37-4285-a864-7208b6c83ef6 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:05.610002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20415f44-43c4-4692-9781-239af89d1e82 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:05.612950 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:05.627726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bdf979-38e8-412f-94e6-bfe1dd602cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:05.628462 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdf979-38e8-412f-94e6-bfe1dd602cde None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:05.628800 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdf979-38e8-412f-94e6-bfe1dd602cde None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:05.629080 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdf979-38e8-412f-94e6-bfe1dd602cde None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:05.629725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bdf979-38e8-412f-94e6-bfe1dd602cde None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:05.630031 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bdf979-38e8-412f-94e6-bfe1dd602cde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:05.631559 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bdf979-38e8-412f-94e6-bfe1dd602cde None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:05.644847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca036da-5963-49b8-8fc4-a3c4f3fc7daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:05.649758 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca036da-5963-49b8-8fc4-a3c4f3fc7daa None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:05.649912 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 342/671] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:05.650007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca036da-5963-49b8-8fc4-a3c4f3fc7daa None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:05.650007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca036da-5963-49b8-8fc4-a3c4f3fc7daa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:05.650193 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bdf979-38e8-412f-94e6-bfe1dd602cde None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:05.661703 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca036da-5963-49b8-8fc4-a3c4f3fc7daa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:05.663133 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca036da-5963-49b8-8fc4-a3c4f3fc7daa None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:05.678928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-2ca036da-5963-49b8-8fc4-a3c4f3fc7daa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:05.678928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:05.678928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:05.678928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:05.678928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:05.678928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:05.678928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:05.678928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:05.678928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:05.678928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:05.678928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:05.678928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:05.678928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:05.678928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:05.678928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:05.678928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:05.679934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:05.679934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:05.679934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:05.679934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:05.679934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:05.679934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:05.679934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:05.679934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:05.679934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:05.679934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:05.679934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:05.679934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:05.679934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:05.679934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:05.679934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:05.679934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:05.679934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:05.680887 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:05.680887 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:05.680887 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:05.680887 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 330/672] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:05.680887 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca036da-5963-49b8-8fc4-a3c4f3fc7daa None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:05.756281 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bdf979-38e8-412f-94e6-bfe1dd602cde None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:05.790819 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a0fa7-5fec-4ff6-b4a8-e5e63ff0a998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:05.791274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca036da-5963-49b8-8fc4-a3c4f3fc7daa None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:05.791493 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a0fa7-5fec-4ff6-b4a8-e5e63ff0a998 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:05.791773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a0fa7-5fec-4ff6-b4a8-e5e63ff0a998 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:05.792043 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a0fa7-5fec-4ff6-b4a8-e5e63ff0a998 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:05.802632 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a0fa7-5fec-4ff6-b4a8-e5e63ff0a998 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:05.804138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a0fa7-5fec-4ff6-b4a8-e5e63ff0a998 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:05.810080 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:05.820566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-f98a0fa7-5fec-4ff6-b4a8-e5e63ff0a998 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:05.820566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:05.820566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:05.820566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:05.820566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:05.820566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:05.820566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:05.820566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:05.820566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:05.820566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:05.820566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:05.820566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:05.820566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:05.820566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:05.820566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:05.820566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:05.821558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:05.821558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:05.821558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:05.821558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:05.821558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:05.821558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:05.821558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:05.821558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:05.821558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:05.821558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:05.821558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:05.821558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:05.821558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:05.821558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:05.821558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:05.821558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:05.821558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:05.822413 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:05.822413 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:05.822413 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:05.822413 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 343/673] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:05.822413 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a0fa7-5fec-4ff6-b4a8-e5e63ff0a998 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:05.836140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d7b67-35db-4287-8b34-b2484eabc4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:05.836755 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d7b67-35db-4287-8b34-b2484eabc4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:05.837037 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d7b67-35db-4287-8b34-b2484eabc4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:05.837313 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d7b67-35db-4287-8b34-b2484eabc4f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:05.838043 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d7b67-35db-4287-8b34-b2484eabc4f8 None admin] RBAC: Adding query filter params (role.id=4680aa64da80470b87075090fca2354b, scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb, user.id=55714a21c1cb4f2b949603230243ed02) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:05.838353 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d7b67-35db-4287-8b34-b2484eabc4f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:05.839823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d7b67-35db-4287-8b34-b2484eabc4f8 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:05.855067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 331/674] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:08:05 2022] GET /identity/v3/role_assignments?user.id=55714a21c1cb4f2b949603230243ed02&scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb&role.id=4680aa64da80470b87075090fca2354b => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:05.855067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d7b67-35db-4287-8b34-b2484eabc4f8 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:05.868734 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:05.945437 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a0fa7-5fec-4ff6-b4a8-e5e63ff0a998 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:05.983684 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d7b67-35db-4287-8b34-b2484eabc4f8 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:05.994870 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69b47a-445f-4d02-bf80-2e79db78c82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:05.995980 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69b47a-445f-4d02-bf80-2e79db78c82c None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:05.996272 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69b47a-445f-4d02-bf80-2e79db78c82c None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:05.997410 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69b47a-445f-4d02-bf80-2e79db78c82c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:06.005578 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de69b47a-445f-4d02-bf80-2e79db78c82c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:06.007069 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de69b47a-445f-4d02-bf80-2e79db78c82c None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:06.020688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-de69b47a-445f-4d02-bf80-2e79db78c82c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:06.020688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:06.020688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:06.020688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:06.020688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:06.020688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:06.020688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:06.020688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:06.020688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:06.020688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:06.020688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:06.020688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:06.020688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:08:06.020688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:08:06.020688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:08:06.021773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:08:06.021773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:06.021773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:06.021773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:06.021773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:06.021773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:06.021773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:06.021773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:06.021773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:06.021773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:06.021773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:06.021773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:06.021773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:06.021773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:06.021773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:06.021773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:06.021773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:06.022770 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:08:06.022770 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:08:06.022770 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:08:06.022770 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:08:06.022770 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:08:06.022770 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:08:06.022770 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:06.022770 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:06.022770 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 344/675] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:06.023323 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69b47a-445f-4d02-bf80-2e79db78c82c None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:06.029567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f403bd-b70a-4b7f-b699-fb8353a537c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:06.030417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f403bd-b70a-4b7f-b699-fb8353a537c9 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:06.030703 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f403bd-b70a-4b7f-b699-fb8353a537c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:06.030974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f403bd-b70a-4b7f-b699-fb8353a537c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:06.031660 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f403bd-b70a-4b7f-b699-fb8353a537c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:06.031965 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f403bd-b70a-4b7f-b699-fb8353a537c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:06.033325 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f403bd-b70a-4b7f-b699-fb8353a537c9 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:06.046079 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 332/676] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:06.047366 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca147902-413b-4fb4-814b-d3ef788b4734 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:06.047652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca147902-413b-4fb4-814b-d3ef788b4734 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:06.047923 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca147902-413b-4fb4-814b-d3ef788b4734 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:06.053246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 333/677] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:06.054052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca147902-413b-4fb4-814b-d3ef788b4734 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:06.107223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:06.134647 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:06.142836 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69b47a-445f-4d02-bf80-2e79db78c82c None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:06.174018 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca147902-413b-4fb4-814b-d3ef788b4734 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:06.181838 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402275a-23cf-4f47-9422-188d9724576c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:06.182529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402275a-23cf-4f47-9422-188d9724576c None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:06.182879 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402275a-23cf-4f47-9422-188d9724576c None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:06.183240 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402275a-23cf-4f47-9422-188d9724576c None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:06.184185 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b402275a-23cf-4f47-9422-188d9724576c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:06.184328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b402275a-23cf-4f47-9422-188d9724576c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:06.185910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b402275a-23cf-4f47-9422-188d9724576c None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:06.197929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 345/678] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:06.204241 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402275a-23cf-4f47-9422-188d9724576c None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:06.213087 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e3f5c5-eec2-4241-ab22-a95722d87e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:06.213698 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3f5c5-eec2-4241-ab22-a95722d87e43 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:06.213975 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3f5c5-eec2-4241-ab22-a95722d87e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:06.214243 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3f5c5-eec2-4241-ab22-a95722d87e43 None admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:06.214938 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e3f5c5-eec2-4241-ab22-a95722d87e43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:06.215273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e3f5c5-eec2-4241-ab22-a95722d87e43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:06.217439 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e3f5c5-eec2-4241-ab22-a95722d87e43 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:06.230086 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 334/679] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:08:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:06.231976 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53543fed-3ea1-49be-bcbe-8c715c6768f4 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:06.232257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53543fed-3ea1-49be-bcbe-8c715c6768f4 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:06.232528 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53543fed-3ea1-49be-bcbe-8c715c6768f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:06.254102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-53543fed-3ea1-49be-bcbe-8c715c6768f4 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:06.255325 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-53543fed-3ea1-49be-bcbe-8c715c6768f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:06.333243 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 335/680] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:08:06.334139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402275a-23cf-4f47-9422-188d9724576c None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:06.334409 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53543fed-3ea1-49be-bcbe-8c715c6768f4 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:06.378104 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:06.389920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6d3ea-e594-4bfc-8b98-b883ef32a0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:06.391278 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6d3ea-e594-4bfc-8b98-b883ef32a0d9 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:06.392348 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6d3ea-e594-4bfc-8b98-b883ef32a0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:06.392838 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6d3ea-e594-4bfc-8b98-b883ef32a0d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:06.404135 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f6d3ea-e594-4bfc-8b98-b883ef32a0d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:06.405475 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f6d3ea-e594-4bfc-8b98-b883ef32a0d9 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:06.421428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-16f6d3ea-e594-4bfc-8b98-b883ef32a0d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:06.421428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:06.421428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:06.421428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:06.421428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:06.421428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:06.421428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:06.421428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:06.421428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:06.421428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:06.421428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:06.421428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:06.421428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:08:06.421428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:08:06.421428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:08:06.422376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:08:06.422376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:06.422376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:06.422376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:06.422376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:06.422376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:06.422376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:06.422376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:06.422376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:06.422376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:06.422376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:06.422376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:06.422376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:06.422376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:06.422376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:06.422376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:06.422376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:06.423299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:08:06.423299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:08:06.423299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:08:06.423299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:08:06.423299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:08:06.423299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:08:06.423299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:06.423299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:06.423299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 346/681] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:06.429368 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6d3ea-e594-4bfc-8b98-b883ef32a0d9 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:06.448468 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:06.475820 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53543fed-3ea1-49be-bcbe-8c715c6768f4 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:06.520635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82785a61-d088-4412-938e-dec99bfd86eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:06.521889 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82785a61-d088-4412-938e-dec99bfd86eb None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:06.522928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82785a61-d088-4412-938e-dec99bfd86eb None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:06.523441 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82785a61-d088-4412-938e-dec99bfd86eb None admin] PATH_INFO: `/v3/users/swift` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:06.533537 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82785a61-d088-4412-938e-dec99bfd86eb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:06.535937 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82785a61-d088-4412-938e-dec99bfd86eb None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:06.555886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-82785a61-d088-4412-938e-dec99bfd86eb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 00:08:06.555886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:06.555886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:06.555886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:06.555886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:06.555886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:06.555886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:06.555886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:06.555886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:06.555886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:06.555886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:06.555886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:06.555886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:08:06.555886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:08:06.555886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:08:06.555886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:08:06.557775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:06.557775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:06.557775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:08:06.557775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:06.557775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:08:06.557775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:06.557775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:06.557775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:06.557775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:06.557775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:06.557775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:06.557775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:06.557775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:06.557775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:06.557775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:06.557775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:06.557775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:06.557775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:06.558944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:06.558944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:06.558944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:08:06.558944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:08:06.558944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:08:06.558944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:08:06.558944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:08:06.558944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:08:06.558944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 00:08:06.558944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:06.566892 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 336/682] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:08:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:06.566892 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82785a61-d088-4412-938e-dec99bfd86eb None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:06.569626 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6d3ea-e594-4bfc-8b98-b883ef32a0d9 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:06.617563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df776e1-0682-46d0-a2b4-3c565300de32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:06.618046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df776e1-0682-46d0-a2b4-3c565300de32 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:06.618490 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df776e1-0682-46d0-a2b4-3c565300de32 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:06.618873 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df776e1-0682-46d0-a2b4-3c565300de32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:06.619651 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df776e1-0682-46d0-a2b4-3c565300de32 None admin] RBAC: Adding query filter params (role.id=4680aa64da80470b87075090fca2354b, scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb, user.id=7e42dfb7d22e4435889cb91327db5477) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:06.619959 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df776e1-0682-46d0-a2b4-3c565300de32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:06.621401 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df776e1-0682-46d0-a2b4-3c565300de32 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:06.632519 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 347/683] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:08:06 2022] GET /identity/v3/role_assignments?user.id=7e42dfb7d22e4435889cb91327db5477&scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb&role.id=4680aa64da80470b87075090fca2354b => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:06.633845 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df776e1-0682-46d0-a2b4-3c565300de32 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:06.690809 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:06.706873 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82785a61-d088-4412-938e-dec99bfd86eb None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:06.754023 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f29043-7f1d-424d-a175-ccbfffde68f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:06.754654 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f29043-7f1d-424d-a175-ccbfffde68f5 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:06.755537 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f29043-7f1d-424d-a175-ccbfffde68f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:06.755829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f29043-7f1d-424d-a175-ccbfffde68f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:06.756554 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f29043-7f1d-424d-a175-ccbfffde68f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:06.756872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f29043-7f1d-424d-a175-ccbfffde68f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:06.758852 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f29043-7f1d-424d-a175-ccbfffde68f5 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:06.760424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:06.782514 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 337/684] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:08:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:06.783902 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f29043-7f1d-424d-a175-ccbfffde68f5 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:06.794998 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df776e1-0682-46d0-a2b4-3c565300de32 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:06.840086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd03e5-9c48-45ea-b839-8f3fd8b189be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:06.841751 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd03e5-9c48-45ea-b839-8f3fd8b189be None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:06.842180 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd03e5-9c48-45ea-b839-8f3fd8b189be None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:06.842461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd03e5-9c48-45ea-b839-8f3fd8b189be None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:06.843884 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd03e5-9c48-45ea-b839-8f3fd8b189be None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:06.844011 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd03e5-9c48-45ea-b839-8f3fd8b189be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:06.848951 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd03e5-9c48-45ea-b839-8f3fd8b189be None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:06.865413 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 348/685] 10.222.0.67 () {56 vars in 1135 bytes} [Wed Aug 31 00:08:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:06.866272 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd03e5-9c48-45ea-b839-8f3fd8b189be None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:06.918204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f29043-7f1d-424d-a175-ccbfffde68f5 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:06.964046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d691ce-f9d6-436b-9ad3-837bc1378196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:06.965084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d691ce-f9d6-436b-9ad3-837bc1378196 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:06.965432 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d691ce-f9d6-436b-9ad3-837bc1378196 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:06.966757 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d691ce-f9d6-436b-9ad3-837bc1378196 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:06.978840 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d691ce-f9d6-436b-9ad3-837bc1378196 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:08:06.986272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-c6d691ce-f9d6-436b-9ad3-837bc1378196 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:08:06.986272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:06.986272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:06.986272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:06.986272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:06.986272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:06.986272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:06.986272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:06.986272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:06.986272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:06.986272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:06.986272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:06.986272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:08:06.986272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:08:06.986272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:08:06.986272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:08:06.987282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:08:06.987282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:08:06.987282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:08:06.987282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:08:06.987282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:06.987282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:06.987282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:06.987282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:06.987282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:06.987282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:06.987282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:06.987282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:06.987282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:06.987282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:06.987282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:06.987282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:06.988116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:06.988116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:06.988116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:06.988116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:06.988116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:08:06.988116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:08:06.988116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:08:06.988116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:08:06.988116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:08:06.988116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:08:06.988116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:08:06.988116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:06.988116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 338/686] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:08:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:06.988823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d691ce-f9d6-436b-9ad3-837bc1378196 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:07.004331 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd03e5-9c48-45ea-b839-8f3fd8b189be None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:07.008855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:07.053840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206c185-6441-40e8-be4d-71563898d88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:07.055327 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206c185-6441-40e8-be4d-71563898d88a None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:07.055840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206c185-6441-40e8-be4d-71563898d88a None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:07.056327 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206c185-6441-40e8-be4d-71563898d88a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:07.057290 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f206c185-6441-40e8-be4d-71563898d88a None admin] RBAC: Adding query filter params (role.id=4680aa64da80470b87075090fca2354b, scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb, user.id=05c0d9554cf94c41a2cd709cab378e53) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:07.058382 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f206c185-6441-40e8-be4d-71563898d88a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:07.060920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f206c185-6441-40e8-be4d-71563898d88a None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:07.065549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:07.074935 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 349/687] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:08:06 2022] GET /identity/v3/role_assignments?user.id=05c0d9554cf94c41a2cd709cab378e53&scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb&role.id=4680aa64da80470b87075090fca2354b => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:07.074935 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206c185-6441-40e8-be4d-71563898d88a None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:07.198174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206c185-6441-40e8-be4d-71563898d88a None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:07.209145 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d691ce-f9d6-436b-9ad3-837bc1378196 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:07.251027 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11af8a0-ced7-4534-8d5a-14042aec22be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:07.251599 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11af8a0-ced7-4534-8d5a-14042aec22be None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:07.251879 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11af8a0-ced7-4534-8d5a-14042aec22be None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:07.252152 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11af8a0-ced7-4534-8d5a-14042aec22be None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:07.252834 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11af8a0-ced7-4534-8d5a-14042aec22be None admin] RBAC: Adding query filter params (name=admin) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:07.253146 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11af8a0-ced7-4534-8d5a-14042aec22be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:07.254535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11af8a0-ced7-4534-8d5a-14042aec22be None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:07.283317 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 350/688] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:08:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:07.286644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11af8a0-ced7-4534-8d5a-14042aec22be None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:07.289831 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709877c-b07e-41c6-bdf5-5a07ec91d1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:07.290708 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709877c-b07e-41c6-bdf5-5a07ec91d1d3 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:07.291242 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709877c-b07e-41c6-bdf5-5a07ec91d1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:07.291729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709877c-b07e-41c6-bdf5-5a07ec91d1d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:07.369561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:07.384155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709877c-b07e-41c6-bdf5-5a07ec91d1d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:07.386407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709877c-b07e-41c6-bdf5-5a07ec91d1d3 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:07.401212 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:07.423927 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11af8a0-ced7-4534-8d5a-14042aec22be None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:07.462824 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3bf149-e32c-476a-8996-11a8135dbc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:07.463070 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bf149-e32c-476a-8996-11a8135dbc55 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:07.463342 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bf149-e32c-476a-8996-11a8135dbc55 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:07.463616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bf149-e32c-476a-8996-11a8135dbc55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:07.480293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3bf149-e32c-476a-8996-11a8135dbc55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:07.482997 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3bf149-e32c-476a-8996-11a8135dbc55 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:07.498802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-2a3bf149-e32c-476a-8996-11a8135dbc55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:07.498802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:07.498802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:07.498802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:07.498802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:07.498802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:07.498802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:07.498802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:07.498802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:07.498802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:07.498802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:07.498802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:07.498802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:07.498802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:07.498802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:07.498802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:07.501013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:07.501013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:07.501013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:07.501013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:07.501013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:07.501013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:07.501013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:07.501013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:07.501013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:07.501013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:07.501013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:07.501013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:07.501013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:07.501013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:07.501013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:07.501013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:07.501013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:07.501957 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:07.501957 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:07.501957 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:07.505020 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 351/689] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:07.506264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3bf149-e32c-476a-8996-11a8135dbc55 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:07.508857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 339/690] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:08:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:08:07.514005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709877c-b07e-41c6-bdf5-5a07ec91d1d3 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:07.630720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3bf149-e32c-476a-8996-11a8135dbc55 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:07.651860 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:07.682101 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb84468-c203-46ca-8723-67918c5120f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:07.684970 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb84468-c203-46ca-8723-67918c5120f8 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:07.687445 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb84468-c203-46ca-8723-67918c5120f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:07.687947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb84468-c203-46ca-8723-67918c5120f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:07.688816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb84468-c203-46ca-8723-67918c5120f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:07.690332 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb84468-c203-46ca-8723-67918c5120f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:07.692057 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb84468-c203-46ca-8723-67918c5120f8 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:07.723235 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 352/691] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:07.726537 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb84468-c203-46ca-8723-67918c5120f8 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:07.728697 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709877c-b07e-41c6-bdf5-5a07ec91d1d3 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:07.756554 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:07.811093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed1e56-3675-4e6e-b37c-dd6a4471e7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:07.812116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed1e56-3675-4e6e-b37c-dd6a4471e7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:07.812615 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed1e56-3675-4e6e-b37c-dd6a4471e7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:07.815050 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed1e56-3675-4e6e-b37c-dd6a4471e7b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:07.837151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed1e56-3675-4e6e-b37c-dd6a4471e7b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:07.842359 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed1e56-3675-4e6e-b37c-dd6a4471e7b6 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:07.870914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-abed1e56-3675-4e6e-b37c-dd6a4471e7b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:07.870914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:07.870914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:07.870914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:07.870914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:07.870914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:07.870914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:07.870914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:07.870914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:07.870914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:07.870914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:07.870914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:07.870914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:07.870914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:07.870914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:07.870914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:07.872113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:07.872113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:07.872113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:07.872113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:07.872113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:07.872113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:07.872113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:07.872113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:07.872113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:07.872113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:07.872113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:07.872113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:07.872113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:07.872113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:07.872113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:07.872113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:07.872113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:07.872982 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:07.872982 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:07.872982 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:07.875602 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 340/692] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:07.883063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed1e56-3675-4e6e-b37c-dd6a4471e7b6 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:07.968379 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb84468-c203-46ca-8723-67918c5120f8 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:08.058844 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9193cf73-442e-4c6d-86c2-906522183416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:08.060079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193cf73-442e-4c6d-86c2-906522183416 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:08.063058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193cf73-442e-4c6d-86c2-906522183416 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:08.063058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193cf73-442e-4c6d-86c2-906522183416 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:08.071337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed1e56-3675-4e6e-b37c-dd6a4471e7b6 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:08.092166 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9193cf73-442e-4c6d-86c2-906522183416 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:08.093920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9193cf73-442e-4c6d-86c2-906522183416 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:08.135411 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-9193cf73-442e-4c6d-86c2-906522183416 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:08:08.135411 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:08.135411 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:08.135411 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:08.135411 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:08.135411 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:08.135411 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:08.135411 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:08.135411 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:08.135411 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:08.135411 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:08.135411 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:08.135411 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:08:08.135411 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:08:08.135411 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:08:08.135411 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:08:08.136480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:08.136480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:08.136480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:08:08.136480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:08.136480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:08:08.136480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:08.136480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:08.136480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:08.136480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:08.136480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:08.136480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:08.136480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:08.136480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:08.136480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:08.136480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:08.136480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:08.136480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:08.136480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:08.137495 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:08.137495 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:08.137495 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:08:08.137495 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:08:08.137495 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:08:08.137495 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:08:08.137495 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:08:08.137495 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:08:08.137495 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:08:08.137495 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:08.139916 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 353/693] 10.222.0.67 () {56 vars in 1093 bytes} [Wed Aug 31 00:08:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:08.147217 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9193cf73-442e-4c6d-86c2-906522183416 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:08.182462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b40f1e-6b22-494e-a033-b1cbcdf1dad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:08.183107 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b40f1e-6b22-494e-a033-b1cbcdf1dad5 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:08.183409 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b40f1e-6b22-494e-a033-b1cbcdf1dad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:08.183670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b40f1e-6b22-494e-a033-b1cbcdf1dad5 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:08.184897 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b40f1e-6b22-494e-a033-b1cbcdf1dad5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:08.188992 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b40f1e-6b22-494e-a033-b1cbcdf1dad5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:08.188992 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b40f1e-6b22-494e-a033-b1cbcdf1dad5 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:08.214762 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 341/694] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:08.217067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7d020-8636-479d-9c49-d9f39861a87d None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:08.217367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7d020-8636-479d-9c49-d9f39861a87d None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:08.217620 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7d020-8636-479d-9c49-d9f39861a87d None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:08.220404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 342/695] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:08.221191 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7d020-8636-479d-9c49-d9f39861a87d None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:08.233995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:08.256157 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:08.316238 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9193cf73-442e-4c6d-86c2-906522183416 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:08.351204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7d020-8636-479d-9c49-d9f39861a87d None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:08.366234 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195d7cd-fa3f-4a0b-889d-44a93586d2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:08.366854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195d7cd-fa3f-4a0b-889d-44a93586d2e1 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:08.367168 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195d7cd-fa3f-4a0b-889d-44a93586d2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:08.367451 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195d7cd-fa3f-4a0b-889d-44a93586d2e1 None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:08.368683 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195d7cd-fa3f-4a0b-889d-44a93586d2e1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:08.368998 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195d7cd-fa3f-4a0b-889d-44a93586d2e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:08.371001 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195d7cd-fa3f-4a0b-889d-44a93586d2e1 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:08.394518 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5bdcf0-f2b1-44c3-8129-b98bb30faacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:08.395887 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5bdcf0-f2b1-44c3-8129-b98bb30faacd None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:08.396018 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 354/696] 10.222.0.67 () {56 vars in 1133 bytes} [Wed Aug 31 00:08:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:08.396487 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5bdcf0-f2b1-44c3-8129-b98bb30faacd None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:08.396955 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5bdcf0-f2b1-44c3-8129-b98bb30faacd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:08.397271 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab9bad-20f1-42f0-8fbf-1457275810b9 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:08.397567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab9bad-20f1-42f0-8fbf-1457275810b9 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:08.397840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab9bad-20f1-42f0-8fbf-1457275810b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:08.411868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5bdcf0-f2b1-44c3-8129-b98bb30faacd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:08.413341 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5bdcf0-f2b1-44c3-8129-b98bb30faacd None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:08.422940 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-ebab9bad-20f1-42f0-8fbf-1457275810b9 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:08.424440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-ebab9bad-20f1-42f0-8fbf-1457275810b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:08.426619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-3c5bdcf0-f2b1-44c3-8129-b98bb30faacd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:08.426619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:08.426619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:08.426619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:08.426619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:08.426619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:08.426619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:08.426619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:08.426619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:08.426619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:08.426619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:08.426619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:08.426619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:08:08.426619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:08:08.426619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:08:08.427491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:08:08.427491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:08.427491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:08.427491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:08.427491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:08.427491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:08.427491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:08.427491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:08.427491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:08.427491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:08.427491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:08.427491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:08.427491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:08.427491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:08.427491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:08.427491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:08.427491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:08.428378 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:08:08.428378 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:08:08.428378 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:08:08.428378 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:08:08.428378 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:08:08.428378 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:08:08.428378 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:08.428378 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:08.431119 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 343/697] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:08.432155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5bdcf0-f2b1-44c3-8129-b98bb30faacd None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:08.527293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 355/698] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:08:08.528726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab9bad-20f1-42f0-8fbf-1457275810b9 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:08.567519 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:08.582069 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:08.593866 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5bdcf0-f2b1-44c3-8129-b98bb30faacd None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:08.667608 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8492a5-e752-40be-951b-3b3ea91f202c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:08.668246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8492a5-e752-40be-951b-3b3ea91f202c None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:08.668527 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8492a5-e752-40be-951b-3b3ea91f202c None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:08.668930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8492a5-e752-40be-951b-3b3ea91f202c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:08.736609 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab9bad-20f1-42f0-8fbf-1457275810b9 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:08.802191 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8492a5-e752-40be-951b-3b3ea91f202c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:08.803689 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8492a5-e752-40be-951b-3b3ea91f202c None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:08.805814 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081eb39-79ae-42a5-8b9a-7439208df0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:08.807508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081eb39-79ae-42a5-8b9a-7439208df0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:08.808018 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081eb39-79ae-42a5-8b9a-7439208df0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:08.808497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081eb39-79ae-42a5-8b9a-7439208df0f4 None admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:08.809936 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1081eb39-79ae-42a5-8b9a-7439208df0f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:08.810559 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1081eb39-79ae-42a5-8b9a-7439208df0f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:08.812647 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1081eb39-79ae-42a5-8b9a-7439208df0f4 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:08.828136 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 356/699] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:08:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:08.829799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d38fe0-df02-41a8-bb4b-e4263d97831b None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:08.830303 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d38fe0-df02-41a8-bb4b-e4263d97831b None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:08.830774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d38fe0-df02-41a8-bb4b-e4263d97831b None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:08.836299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 357/700] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:08.836299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d38fe0-df02-41a8-bb4b-e4263d97831b None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:08.901463 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:08.926140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 344/701] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:08:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:08:08.926140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267ad65-1400-411f-a2ab-214caf06776e None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:08.926140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267ad65-1400-411f-a2ab-214caf06776e None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:08.926140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267ad65-1400-411f-a2ab-214caf06776e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:08.939287 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:08.944960 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-3267ad65-1400-411f-a2ab-214caf06776e None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:08.946154 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-3267ad65-1400-411f-a2ab-214caf06776e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:08.964088 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d38fe0-df02-41a8-bb4b-e4263d97831b None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:09.011625 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c5ca7-91cc-47a0-bb8e-8c270e80a26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:09.012355 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c5ca7-91cc-47a0-bb8e-8c270e80a26c None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:09.012355 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c5ca7-91cc-47a0-bb8e-8c270e80a26c None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:09.013602 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c5ca7-91cc-47a0-bb8e-8c270e80a26c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:09.013982 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c5ca7-91cc-47a0-bb8e-8c270e80a26c None admin] RBAC: Adding query filter params (role.id=4680aa64da80470b87075090fca2354b, scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb, user.id=1cd3cfbc8ebd4c5a83bef09efe10bc91) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:09.014301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c5ca7-91cc-47a0-bb8e-8c270e80a26c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:09.015941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c5ca7-91cc-47a0-bb8e-8c270e80a26c None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:09.027114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 345/702] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:08:09.027114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267ad65-1400-411f-a2ab-214caf06776e None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:09.035266 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 358/703] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:08:08 2022] GET /identity/v3/role_assignments?user.id=1cd3cfbc8ebd4c5a83bef09efe10bc91&scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb&role.id=4680aa64da80470b87075090fca2354b => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:09.036277 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c5ca7-91cc-47a0-bb8e-8c270e80a26c None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:09.179515 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267ad65-1400-411f-a2ab-214caf06776e None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:09.193637 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:09.216613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd4527-7054-45e5-a5ae-331eecb6d130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:09.217417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd4527-7054-45e5-a5ae-331eecb6d130 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:09.217790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd4527-7054-45e5-a5ae-331eecb6d130 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:09.218157 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd4527-7054-45e5-a5ae-331eecb6d130 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:09.232383 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd4527-7054-45e5-a5ae-331eecb6d130 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:09.233831 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd4527-7054-45e5-a5ae-331eecb6d130 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:09.243279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c5ca7-91cc-47a0-bb8e-8c270e80a26c None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:09.251790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-23cd4527-7054-45e5-a5ae-331eecb6d130 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:09.251790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:09.251790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:09.251790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:09.251790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:09.251790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:09.251790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:09.251790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:09.251790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:09.251790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:09.251790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:09.251790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:09.251790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:09.251790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:09.251790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:09.251790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:09.253005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:09.253005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:09.253005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:09.253005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:09.253005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:09.253005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:09.253005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:09.253005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:09.253005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:09.253005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:09.253005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:09.253005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:09.253005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:09.253005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:09.253005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:09.253005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:09.253005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:09.253935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:09.253935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:09.253935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:09.253935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 346/704] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:09.258707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd4527-7054-45e5-a5ae-331eecb6d130 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:09.295543 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cbe110-6ffd-4ff7-9e8a-47dc144de164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:09.296458 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbe110-6ffd-4ff7-9e8a-47dc144de164 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:09.297306 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbe110-6ffd-4ff7-9e8a-47dc144de164 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:09.297591 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbe110-6ffd-4ff7-9e8a-47dc144de164 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:09.312715 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cbe110-6ffd-4ff7-9e8a-47dc144de164 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:09.314908 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cbe110-6ffd-4ff7-9e8a-47dc144de164 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:09.323785 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:09.336782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-14cbe110-6ffd-4ff7-9e8a-47dc144de164 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:09.336782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:09.336782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:09.336782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:09.336782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:09.336782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:09.336782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:09.336782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:09.336782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:09.336782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:09.336782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:09.336782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:09.336782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:09.336782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:09.336782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:09.336782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:09.337787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:09.337787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:09.337787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:09.337787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:09.337787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:09.337787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:09.337787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:09.337787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:09.337787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:09.337787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:09.337787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:09.337787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:09.337787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:09.337787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:09.337787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:09.337787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:09.337787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:09.338690 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:09.338690 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:09.338690 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:09.338859 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 359/705] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:09.344343 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cbe110-6ffd-4ff7-9e8a-47dc144de164 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:09.398482 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd4527-7054-45e5-a5ae-331eecb6d130 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:09.443829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedaae1f-b4c6-48c0-aa75-0ab2cef609ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:09.444334 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedaae1f-b4c6-48c0-aa75-0ab2cef609ba None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:09.444616 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedaae1f-b4c6-48c0-aa75-0ab2cef609ba None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:09.444896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedaae1f-b4c6-48c0-aa75-0ab2cef609ba None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:09.445544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedaae1f-b4c6-48c0-aa75-0ab2cef609ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:09.446592 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedaae1f-b4c6-48c0-aa75-0ab2cef609ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:09.448127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedaae1f-b4c6-48c0-aa75-0ab2cef609ba None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:09.467200 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 347/706] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:09.468096 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedaae1f-b4c6-48c0-aa75-0ab2cef609ba None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:09.495763 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cbe110-6ffd-4ff7-9e8a-47dc144de164 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:09.499718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:09.549323 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422777c-ff10-4a76-b0cf-e94b1404a3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:09.550234 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422777c-ff10-4a76-b0cf-e94b1404a3f9 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:09.550522 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422777c-ff10-4a76-b0cf-e94b1404a3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:09.550801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422777c-ff10-4a76-b0cf-e94b1404a3f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:09.551771 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422777c-ff10-4a76-b0cf-e94b1404a3f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:09.552084 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422777c-ff10-4a76-b0cf-e94b1404a3f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:09.554529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422777c-ff10-4a76-b0cf-e94b1404a3f9 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:09.569541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 360/707] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:09.570370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422777c-ff10-4a76-b0cf-e94b1404a3f9 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:09.584659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedaae1f-b4c6-48c0-aa75-0ab2cef609ba None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:09.627176 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20e714-b79f-4efd-bf6a-956d52be38d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:09.627815 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20e714-b79f-4efd-bf6a-956d52be38d5 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:09.628107 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20e714-b79f-4efd-bf6a-956d52be38d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:09.628368 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20e714-b79f-4efd-bf6a-956d52be38d5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:09.628954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f20e714-b79f-4efd-bf6a-956d52be38d5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:09.630307 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f20e714-b79f-4efd-bf6a-956d52be38d5 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:09.632394 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:09.641572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-8f20e714-b79f-4efd-bf6a-956d52be38d5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 00:08:09.641572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:09.641572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:08:09.641572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:08:09.641572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:08:09.641572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:08:09.641572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:08:09.641572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:09.641572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 00:08:09.641572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:09.641572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:08:09.641572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:09.641572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:09.641572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:09.641572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:09.641572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:09.641572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:09.642634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:09.642634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:09.642634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:09.642634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:09.642634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:09.642634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:09.642634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:08:09.642634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:08:09.642634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:08:09.642634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:08:09.642634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:09.642634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:09.642634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:09.642634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:09.642634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:09.642634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:09.643958 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:09.643958 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:09.643958 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:09.643958 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:09.643958 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:09.643958 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:09.643958 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:09.643958 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:09.643958 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:09.643958 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:09.643958 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:08:09.643958 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:09.643958 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 00:08:09.643958 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:09.643958 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 348/708] 10.222.0.67 () {56 vars in 1129 bytes} [Wed Aug 31 00:08:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:09.644783 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78278311-7b12-4a14-8b35-5d98222ecdaf None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:09.644783 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78278311-7b12-4a14-8b35-5d98222ecdaf None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:09.644783 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78278311-7b12-4a14-8b35-5d98222ecdaf None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:09.650217 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 349/709] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:09.650217 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78278311-7b12-4a14-8b35-5d98222ecdaf None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:09.689546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422777c-ff10-4a76-b0cf-e94b1404a3f9 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:09.727496 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb76bd-6508-45f2-ae4e-3ef05b66af70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:09.728515 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb76bd-6508-45f2-ae4e-3ef05b66af70 None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:09.728515 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb76bd-6508-45f2-ae4e-3ef05b66af70 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:09.728708 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb76bd-6508-45f2-ae4e-3ef05b66af70 None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:09.729192 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb76bd-6508-45f2-ae4e-3ef05b66af70 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:09.730620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb76bd-6508-45f2-ae4e-3ef05b66af70 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:09.749510 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:09.753048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 361/710] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 31 00:08:09 2022] POST /identity/v3/users => generated 254 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:08:09.753749 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb76bd-6508-45f2-ae4e-3ef05b66af70 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:09.781057 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78278311-7b12-4a14-8b35-5d98222ecdaf None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:09.835648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa5b4f-e7ca-4400-99e6-991b5d9cce78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:09.836308 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa5b4f-e7ca-4400-99e6-991b5d9cce78 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:09.836905 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa5b4f-e7ca-4400-99e6-991b5d9cce78 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:09.836905 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa5b4f-e7ca-4400-99e6-991b5d9cce78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:09.846375 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa5b4f-e7ca-4400-99e6-991b5d9cce78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:09.847892 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa5b4f-e7ca-4400-99e6-991b5d9cce78 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:09.860490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-b8fa5b4f-e7ca-4400-99e6-991b5d9cce78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:09.860490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:09.860490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:09.860490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:09.860490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:09.860490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:09.860490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:09.860490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:09.860490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:09.860490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:09.860490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:09.860490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:09.860490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:08:09.860490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:08:09.860490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:08:09.861476 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:08:09.861476 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:09.861476 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:09.861476 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:09.861476 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:09.861476 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:09.861476 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:09.861476 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:09.861476 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:09.861476 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:09.861476 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:09.861476 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:09.861476 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:09.861476 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:09.861476 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:09.861476 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:09.861476 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:09.862379 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:08:09.862379 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:08:09.862379 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:08:09.862379 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:08:09.862379 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:08:09.862379 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:08:09.862379 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:09.862379 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:09.866770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 350/711] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:09.868283 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2b5bb-6fe6-416c-b40b-e4f2c8c38e87 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:09.868576 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2b5bb-6fe6-416c-b40b-e4f2c8c38e87 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:09.868949 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2b5bb-6fe6-416c-b40b-e4f2c8c38e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:09.889947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-a0a2b5bb-6fe6-416c-b40b-e4f2c8c38e87 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:09.891213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-a0a2b5bb-6fe6-416c-b40b-e4f2c8c38e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:09.895009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb76bd-6508-45f2-ae4e-3ef05b66af70 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:09.911348 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:09.938617 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e89694-708d-40ae-ba19-764d7ceb2beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:09.940042 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e89694-708d-40ae-ba19-764d7ceb2beb None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:09.940555 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e89694-708d-40ae-ba19-764d7ceb2beb None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:09.940889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e89694-708d-40ae-ba19-764d7ceb2beb None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:09.942289 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e89694-708d-40ae-ba19-764d7ceb2beb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:09.942289 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e89694-708d-40ae-ba19-764d7ceb2beb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:09.944750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e89694-708d-40ae-ba19-764d7ceb2beb None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:09.950780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 362/712] 10.222.0.67 () {56 vars in 1124 bytes} [Wed Aug 31 00:08:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:09.951583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e89694-708d-40ae-ba19-764d7ceb2beb None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:09.971746 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 351/713] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:08:09.972186 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2b5bb-6fe6-416c-b40b-e4f2c8c38e87 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:10.065229 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:10.089222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2b5bb-6fe6-416c-b40b-e4f2c8c38e87 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:10.110187 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e89694-708d-40ae-ba19-764d7ceb2beb None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:10.127899 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13c150-1ac4-49a2-a749-d8197cc30df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:10.128608 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13c150-1ac4-49a2-a749-d8197cc30df4 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:10.128957 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13c150-1ac4-49a2-a749-d8197cc30df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:10.129302 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13c150-1ac4-49a2-a749-d8197cc30df4 None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:10.130000 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d13c150-1ac4-49a2-a749-d8197cc30df4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:10.130371 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d13c150-1ac4-49a2-a749-d8197cc30df4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:10.131859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d13c150-1ac4-49a2-a749-d8197cc30df4 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:10.138589 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 352/714] 10.222.0.67 () {56 vars in 1124 bytes} [Wed Aug 31 00:08:09 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:10.153502 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa31b60-46ac-429c-bd3a-14e5303f8eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:10.154143 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa31b60-46ac-429c-bd3a-14e5303f8eda None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:10.154466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa31b60-46ac-429c-bd3a-14e5303f8eda None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:10.154745 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa31b60-46ac-429c-bd3a-14e5303f8eda None admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:10.155491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa31b60-46ac-429c-bd3a-14e5303f8eda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:10.155798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa31b60-46ac-429c-bd3a-14e5303f8eda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:10.157390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa31b60-46ac-429c-bd3a-14e5303f8eda None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:10.173027 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 363/715] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:08:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:10.178613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13c150-1ac4-49a2-a749-d8197cc30df4 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:10.333980 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13c150-1ac4-49a2-a749-d8197cc30df4 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:10.343585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa31b60-46ac-429c-bd3a-14e5303f8eda None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:10.373379 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eb5626-baf8-4677-95c5-5f279b9e3db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:10.374410 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb5626-baf8-4677-95c5-5f279b9e3db1 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:10.374906 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb5626-baf8-4677-95c5-5f279b9e3db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:10.375672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb5626-baf8-4677-95c5-5f279b9e3db1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:10.376619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eb5626-baf8-4677-95c5-5f279b9e3db1 None admin] RBAC: Adding query filter params (role.id=34df06ce84ad4cf8b3bd1a4490228017, scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb, user.id=3aa53ae5a30146f69d7d6eefe4290dba) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:10.377156 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eb5626-baf8-4677-95c5-5f279b9e3db1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:10.378848 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eb5626-baf8-4677-95c5-5f279b9e3db1 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:10.385508 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 353/716] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:08:10 2022] GET /identity/v3/role_assignments?user.id=3aa53ae5a30146f69d7d6eefe4290dba&scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb&role.id=34df06ce84ad4cf8b3bd1a4490228017 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:10.404838 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:10.484885 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa31b60-46ac-429c-bd3a-14e5303f8eda None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:10.526569 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0b6eb-bbaa-4458-a4e9-55e6f9cdcb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:10.527248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0b6eb-bbaa-4458-a4e9-55e6f9cdcb87 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:10.527528 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0b6eb-bbaa-4458-a4e9-55e6f9cdcb87 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:10.527803 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0b6eb-bbaa-4458-a4e9-55e6f9cdcb87 None admin] PATH_INFO: `/v3/services/network` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:10.528442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0b6eb-bbaa-4458-a4e9-55e6f9cdcb87 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:10.529847 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0b6eb-bbaa-4458-a4e9-55e6f9cdcb87 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:10.541565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-72c0b6eb-bbaa-4458-a4e9-55e6f9cdcb87 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 00:08:10.541565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:10.541565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:08:10.541565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:08:10.541565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:08:10.541565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:08:10.541565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:08:10.541565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:10.541565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 00:08:10.541565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:10.541565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:08:10.541565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:10.541565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:10.541565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:10.541565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:10.541565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:10.541565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:10.542597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:10.542597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:10.542597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:10.542597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:10.542597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:10.542597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:10.542597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:08:10.542597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:08:10.542597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:08:10.542597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:08:10.542597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:10.542597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:10.542597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:10.542597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:10.542597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:10.542597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:10.543489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:10.543489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:10.543489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:10.543489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:10.543489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:10.543489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:10.543489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:10.543489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:10.543489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:10.543489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:10.543489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:08:10.543489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:10.543489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 00:08:10.543489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:10.547176 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 364/717] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:10 2022] GET /identity/v3/services/network => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:10.551262 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eb5626-baf8-4677-95c5-5f279b9e3db1 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:10.566626 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:10.665459 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eb5626-baf8-4677-95c5-5f279b9e3db1 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:10.701988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541dbf39-cee1-48ee-9bf8-58cf67bdf722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:10.702982 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541dbf39-cee1-48ee-9bf8-58cf67bdf722 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:10.703542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541dbf39-cee1-48ee-9bf8-58cf67bdf722 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:10.704016 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541dbf39-cee1-48ee-9bf8-58cf67bdf722 None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:10.704858 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541dbf39-cee1-48ee-9bf8-58cf67bdf722 None admin] RBAC: Adding query filter params (name=network) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:10.705367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541dbf39-cee1-48ee-9bf8-58cf67bdf722 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:10.706934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541dbf39-cee1-48ee-9bf8-58cf67bdf722 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:10.712334 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 354/718] 10.222.0.67 () {56 vars in 1112 bytes} [Wed Aug 31 00:08:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:10.720147 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0b6eb-bbaa-4458-a4e9-55e6f9cdcb87 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:10.861851 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0b6eb-bbaa-4458-a4e9-55e6f9cdcb87 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:10.863418 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:10.910535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14980b87-7d7d-4e32-9ebc-9bce611f04ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:10.911185 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14980b87-7d7d-4e32-9ebc-9bce611f04ff None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:10.911468 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14980b87-7d7d-4e32-9ebc-9bce611f04ff None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:10.911740 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14980b87-7d7d-4e32-9ebc-9bce611f04ff None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:10.912374 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14980b87-7d7d-4e32-9ebc-9bce611f04ff None admin] RBAC: Adding query filter params (type=network) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:10.912678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14980b87-7d7d-4e32-9ebc-9bce611f04ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:10.914063 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14980b87-7d7d-4e32-9ebc-9bce611f04ff None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:10.920789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 365/719] 10.222.0.67 () {56 vars in 1112 bytes} [Wed Aug 31 00:08:10 2022] GET /identity/v3/services?type=network => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:11.302390 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45818c-fadd-4fff-889f-78ba8455ac9b None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:11.302611 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45818c-fadd-4fff-889f-78ba8455ac9b None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:11.302969 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45818c-fadd-4fff-889f-78ba8455ac9b None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:11.307369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 355/720] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:11.313880 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bcd50-8fc8-4c09-8581-2f2354605bb0 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:11.314189 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bcd50-8fc8-4c09-8581-2f2354605bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:11.314435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bcd50-8fc8-4c09-8581-2f2354605bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:11.333312 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-e00bcd50-8fc8-4c09-8581-2f2354605bb0 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:11.334494 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e00bcd50-8fc8-4c09-8581-2f2354605bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:11.428331 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 366/721] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:08:11.808865 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45818c-fadd-4fff-889f-78ba8455ac9b None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:11.899473 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:11.944686 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45818c-fadd-4fff-889f-78ba8455ac9b None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:11.987452 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27428eb2-62d6-4478-b089-bc6e72835221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:11.988128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27428eb2-62d6-4478-b089-bc6e72835221 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:11.988389 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27428eb2-62d6-4478-b089-bc6e72835221 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:11.988633 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27428eb2-62d6-4478-b089-bc6e72835221 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:12.019961 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27428eb2-62d6-4478-b089-bc6e72835221 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:08:12.025257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-27428eb2-62d6-4478-b089-bc6e72835221 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:08:12.025257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:12.025257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:12.025257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:12.025257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:12.025257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:12.025257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:12.025257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:12.025257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:12.025257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:12.025257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:12.025257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:12.025257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:08:12.025257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:08:12.025257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:08:12.025257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:08:12.026170 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:08:12.026170 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:08:12.026170 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:08:12.026170 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:08:12.026170 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:12.026170 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:12.026170 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:12.026170 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:12.026170 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:12.026170 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:12.026170 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:12.026170 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:12.026170 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:12.026170 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:12.026170 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:12.026170 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:12.027613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:12.027613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:12.027613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:12.027613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:12.027613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:08:12.027613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:08:12.027613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:08:12.027613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:08:12.027613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:08:12.027613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:08:12.027613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:08:12.027613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:12.027613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 356/722] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:08:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:12.032999 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00bcd50-8fc8-4c09-8581-2f2354605bb0 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:12.201908 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:12.207632 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00bcd50-8fc8-4c09-8581-2f2354605bb0 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:12.261530 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24876285-4d48-4bdb-a9bc-5f3ae0b70ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:12.262148 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24876285-4d48-4bdb-a9bc-5f3ae0b70ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:12.262959 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24876285-4d48-4bdb-a9bc-5f3ae0b70ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:12.263254 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24876285-4d48-4bdb-a9bc-5f3ae0b70ba1 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:12.264115 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24876285-4d48-4bdb-a9bc-5f3ae0b70ba1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:12.264418 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24876285-4d48-4bdb-a9bc-5f3ae0b70ba1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:12.265803 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24876285-4d48-4bdb-a9bc-5f3ae0b70ba1 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:12.278965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 367/723] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:08:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:12.284150 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27428eb2-62d6-4478-b089-bc6e72835221 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:12.441859 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bab218-195c-4cba-8740-2a00b48cbc84 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:12.442233 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bab218-195c-4cba-8740-2a00b48cbc84 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:12.442415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bab218-195c-4cba-8740-2a00b48cbc84 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:12.443907 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 368/724] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:08:12.451391 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d5571-66e6-412d-aafd-a4ed7f664a51 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:12.452073 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d5571-66e6-412d-aafd-a4ed7f664a51 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:12.452569 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d5571-66e6-412d-aafd-a4ed7f664a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:12.469213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27428eb2-62d6-4478-b089-bc6e72835221 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:12.474819 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-ef7d5571-66e6-412d-aafd-a4ed7f664a51 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:12.477512 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-ef7d5571-66e6-412d-aafd-a4ed7f664a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:12.548873 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:12.560668 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 369/725] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:08:12.567782 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d3cbb-8941-4c5e-8c6d-44656f54f338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:12.568445 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d3cbb-8941-4c5e-8c6d-44656f54f338 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:12.568724 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d3cbb-8941-4c5e-8c6d-44656f54f338 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:12.569016 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d3cbb-8941-4c5e-8c6d-44656f54f338 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:12.592042 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d3cbb-8941-4c5e-8c6d-44656f54f338 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:12.593513 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d3cbb-8941-4c5e-8c6d-44656f54f338 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:12.609391 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30d1c8-97c1-4a1a-b7c6-c64323527f82 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:12.609693 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30d1c8-97c1-4a1a-b7c6-c64323527f82 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:12.609942 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30d1c8-97c1-4a1a-b7c6-c64323527f82 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:12.612523 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 370/726] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:08:12.616210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-f76d3cbb-8941-4c5e-8c6d-44656f54f338 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:12.616210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:12.616210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:12.616210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:12.616210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:12.616210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:12.616210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:12.616210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:12.616210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:12.616210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:12.616210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:12.616210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:12.616210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:12.616210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:12.616210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:12.616210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:12.617139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:12.617139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:12.617139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:12.617139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:12.617139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:12.617139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:12.617139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:12.617139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:12.617139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:12.617139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:12.617139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:12.617139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:12.617139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:12.617139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:12.617139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:12.617139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:12.617139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:12.618058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:12.618058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:12.618058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:12.618228 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bc682-01c8-4150-8cc1-7f390e9625e8 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:12.618228 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bc682-01c8-4150-8cc1-7f390e9625e8 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:12.618228 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bc682-01c8-4150-8cc1-7f390e9625e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:12.618429 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 357/727] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:12.623878 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d3cbb-8941-4c5e-8c6d-44656f54f338 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:12.644965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-776bc682-01c8-4150-8cc1-7f390e9625e8 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:12.646319 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-776bc682-01c8-4150-8cc1-7f390e9625e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:12.761171 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d3cbb-8941-4c5e-8c6d-44656f54f338 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:12.785621 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 371/728] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 00:08:12.797611 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98b77d-df63-458a-9343-ddbbc827fb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:12.798505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98b77d-df63-458a-9343-ddbbc827fb04 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:12.799015 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98b77d-df63-458a-9343-ddbbc827fb04 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:12.799528 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98b77d-df63-458a-9343-ddbbc827fb04 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:12.800380 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d98b77d-df63-458a-9343-ddbbc827fb04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:12.800884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d98b77d-df63-458a-9343-ddbbc827fb04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:12.802436 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d98b77d-df63-458a-9343-ddbbc827fb04 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:12.815642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 358/729] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:12.824306 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776bc682-01c8-4150-8cc1-7f390e9625e8 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:12.859399 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:12.945413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98b77d-df63-458a-9343-ddbbc827fb04 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:12.995442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:12.999760 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776bc682-01c8-4150-8cc1-7f390e9625e8 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:13.050579 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266951ac-f199-40df-ab89-a6952ff7ce5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:13.051219 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266951ac-f199-40df-ab89-a6952ff7ce5a None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:13.051464 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266951ac-f199-40df-ab89-a6952ff7ce5a None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:13.051713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266951ac-f199-40df-ab89-a6952ff7ce5a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:13.059546 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98b77d-df63-458a-9343-ddbbc827fb04 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:13.059995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266951ac-f199-40df-ab89-a6952ff7ce5a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:13.061458 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266951ac-f199-40df-ab89-a6952ff7ce5a None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:13.073913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-266951ac-f199-40df-ab89-a6952ff7ce5a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 00:08:13.073913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:13.073913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:13.073913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:13.073913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:13.073913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:13.073913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:13.073913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:13.073913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:13.073913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:13.073913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:13.073913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:13.073913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:08:13.073913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:08:13.073913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:08:13.073913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:08:13.075557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:13.075557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:13.075557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:08:13.075557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:13.075557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:08:13.075557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:13.075557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:13.075557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:13.075557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:13.075557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:13.075557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:13.075557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:13.075557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:13.075557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:13.075557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:13.075557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:13.075557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:13.075557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:13.076883 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:13.076883 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:13.076883 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:08:13.076883 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:08:13.076883 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:08:13.076883 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:08:13.076883 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:08:13.076883 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:08:13.076883 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 00:08:13.076883 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:13.076883 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 372/730] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:08:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:13.080757 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88fc93-c970-4486-9558-378794abbd26 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:13.081287 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88fc93-c970-4486-9558-378794abbd26 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:13.081764 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88fc93-c970-4486-9558-378794abbd26 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:13.083838 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 373/731] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:13.084943 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88fc93-c970-4486-9558-378794abbd26 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:13.111948 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5301cbae-24e3-49c3-8ed0-32f6978f795e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:13.112590 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301cbae-24e3-49c3-8ed0-32f6978f795e None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:13.112867 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301cbae-24e3-49c3-8ed0-32f6978f795e None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:13.113138 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301cbae-24e3-49c3-8ed0-32f6978f795e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:13.127970 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5301cbae-24e3-49c3-8ed0-32f6978f795e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:08:13.133890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-5301cbae-24e3-49c3-8ed0-32f6978f795e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:08:13.133890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:13.133890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:13.133890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:13.133890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:13.133890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:13.133890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:13.133890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:13.133890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:13.133890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:13.133890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:13.133890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:13.133890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:08:13.133890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:08:13.133890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:08:13.133890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:08:13.134847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:08:13.134847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:08:13.134847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:08:13.134847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:08:13.134847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:13.134847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:13.134847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:13.134847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:13.134847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:13.134847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:13.134847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:13.134847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:13.134847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:13.134847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:13.134847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:13.134847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:13.136137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:13.136137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:13.136137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:13.136137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:13.136137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:08:13.136137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:08:13.136137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:08:13.136137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:08:13.136137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:08:13.136137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:08:13.136137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:08:13.136137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:13.136137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 359/732] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 31 00:08:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:13.137070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52ff32-6156-4d5b-bf92-f5d568a0de75 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:13.137070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52ff32-6156-4d5b-bf92-f5d568a0de75 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:13.137070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52ff32-6156-4d5b-bf92-f5d568a0de75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:13.157669 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-2d52ff32-6156-4d5b-bf92-f5d568a0de75 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:13.159090 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-2d52ff32-6156-4d5b-bf92-f5d568a0de75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:13.183423 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:13.208644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88fc93-c970-4486-9558-378794abbd26 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:13.234235 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 360/733] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:08:13.235096 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52ff32-6156-4d5b-bf92-f5d568a0de75 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:13.254111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e7ba9-22b5-45e4-a7cb-3d71859f9b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:13.255542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e7ba9-22b5-45e4-a7cb-3d71859f9b5b None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:13.256050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e7ba9-22b5-45e4-a7cb-3d71859f9b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:13.256554 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e7ba9-22b5-45e4-a7cb-3d71859f9b5b None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:13.258124 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e7ba9-22b5-45e4-a7cb-3d71859f9b5b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:13.258644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e7ba9-22b5-45e4-a7cb-3d71859f9b5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:13.260904 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e7ba9-22b5-45e4-a7cb-3d71859f9b5b None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:13.268027 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:13.293024 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 374/734] 10.222.0.67 () {56 vars in 1135 bytes} [Wed Aug 31 00:08:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:13.293024 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e7ba9-22b5-45e4-a7cb-3d71859f9b5b None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:13.351757 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52ff32-6156-4d5b-bf92-f5d568a0de75 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:13.399988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6a664-adda-4179-81b5-28c3243fe0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:13.400808 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6a664-adda-4179-81b5-28c3243fe0df None admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:13.401191 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6a664-adda-4179-81b5-28c3243fe0df None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:13.401562 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6a664-adda-4179-81b5-28c3243fe0df None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:13.402335 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6a664-adda-4179-81b5-28c3243fe0df None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:13.404374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6a664-adda-4179-81b5-28c3243fe0df None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:13.423980 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e7ba9-22b5-45e4-a7cb-3d71859f9b5b None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:13.429625 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 361/735] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 31 00:08:13 2022] POST /identity/v3/services => generated 251 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:08:13.430660 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6a664-adda-4179-81b5-28c3243fe0df None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:13.471547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:13.477685 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41c1d5-7ad1-4d0e-bcbc-d3f3c3e49ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:13.479233 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41c1d5-7ad1-4d0e-bcbc-d3f3c3e49ecb None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:13.479756 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41c1d5-7ad1-4d0e-bcbc-d3f3c3e49ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:13.480266 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41c1d5-7ad1-4d0e-bcbc-d3f3c3e49ecb None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:13.481886 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce41c1d5-7ad1-4d0e-bcbc-d3f3c3e49ecb None admin] RBAC: Adding query filter params (name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:13.482429 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce41c1d5-7ad1-4d0e-bcbc-d3f3c3e49ecb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:13.484229 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce41c1d5-7ad1-4d0e-bcbc-d3f3c3e49ecb None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:13.494373 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 375/736] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 31 00:08:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:13.501236 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41c1d5-7ad1-4d0e-bcbc-d3f3c3e49ecb None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:13.561574 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:13.589905 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6a664-adda-4179-81b5-28c3243fe0df None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:13.635847 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41c1d5-7ad1-4d0e-bcbc-d3f3c3e49ecb None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:13.636985 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b2139-f73c-4614-b47a-9a1a7bec1231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:13.638268 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b2139-f73c-4614-b47a-9a1a7bec1231 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:13.638727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b2139-f73c-4614-b47a-9a1a7bec1231 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:13.639243 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b2139-f73c-4614-b47a-9a1a7bec1231 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:13.679898 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea042ac-40df-4ec7-9f95-592662e4a710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:13.680568 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea042ac-40df-4ec7-9f95-592662e4a710 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:13.680851 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea042ac-40df-4ec7-9f95-592662e4a710 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:13.681129 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea042ac-40df-4ec7-9f95-592662e4a710 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:13.694891 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea042ac-40df-4ec7-9f95-592662e4a710 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:13.696397 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea042ac-40df-4ec7-9f95-592662e4a710 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:13.714265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-5ea042ac-40df-4ec7-9f95-592662e4a710 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:13.714265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:13.714265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:13.714265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:13.714265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:13.714265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:13.714265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:13.714265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:13.714265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:13.714265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:13.714265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:13.714265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:13.714265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:13.714265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:13.714265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:13.714265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:13.715369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:13.715369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:13.715369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:13.715369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:13.715369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:13.715369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:13.715369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:13.715369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:13.715369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:13.715369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:13.715369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:13.715369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:13.715369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:13.715369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:13.715369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:13.715369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:13.715369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:13.717112 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:13.717112 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:13.717112 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:13.717281 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b2139-f73c-4614-b47a-9a1a7bec1231 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:13.717356 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 376/737] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:13.717432 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b2139-f73c-4614-b47a-9a1a7bec1231 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:13.717663 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2775cc-087e-4761-b782-d8c531e3b814 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:13.717908 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2775cc-087e-4761-b782-d8c531e3b814 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:13.718190 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2775cc-087e-4761-b782-d8c531e3b814 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:13.721558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 377/738] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:13.721558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2775cc-087e-4761-b782-d8c531e3b814 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:13.781108 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:13.819723 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:13.830937 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 362/739] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:08:13 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 00:08:13.831630 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b2139-f73c-4614-b47a-9a1a7bec1231 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:13.848930 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2775cc-087e-4761-b782-d8c531e3b814 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:13.888326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1a8a6-0ecb-47cb-8ad5-e12cbe0f584f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:13.889133 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1a8a6-0ecb-47cb-8ad5-e12cbe0f584f None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:13.889415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1a8a6-0ecb-47cb-8ad5-e12cbe0f584f None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:13.889688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1a8a6-0ecb-47cb-8ad5-e12cbe0f584f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:13.900969 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c1a8a6-0ecb-47cb-8ad5-e12cbe0f584f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:08:13.905941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-74c1a8a6-0ecb-47cb-8ad5-e12cbe0f584f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:08:13.905941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:13.905941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:13.905941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:13.905941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:13.905941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:13.905941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:13.905941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:13.905941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:13.905941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:13.905941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:13.905941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:13.905941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:08:13.905941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:08:13.905941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:08:13.905941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:08:13.906889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:08:13.906889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:08:13.906889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:08:13.906889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:08:13.906889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:13.906889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:13.906889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:13.906889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:13.906889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:13.906889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:13.906889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:13.906889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:13.906889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:13.906889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:13.906889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:13.906889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:13.907772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:13.907772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:13.907772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:13.907772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:13.907772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:08:13.907772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:08:13.907772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:08:13.907772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:08:13.907772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:08:13.907772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:08:13.907772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:08:13.907772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:13.907772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 378/740] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:08:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:13.908488 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78394302-fd4d-42e9-835a-2be362a92894 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:13.908488 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78394302-fd4d-42e9-835a-2be362a92894 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:13.909246 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78394302-fd4d-42e9-835a-2be362a92894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:13.930561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-78394302-fd4d-42e9-835a-2be362a92894 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:13.931780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-78394302-fd4d-42e9-835a-2be362a92894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:13.946114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b2139-f73c-4614-b47a-9a1a7bec1231 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:13.990901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33514d-e209-48ac-b474-52951a20fbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:13.990901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33514d-e209-48ac-b474-52951a20fbc4 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:13.990901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33514d-e209-48ac-b474-52951a20fbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:13.990901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33514d-e209-48ac-b474-52951a20fbc4 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:13.991410 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e33514d-e209-48ac-b474-52951a20fbc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:13.991410 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e33514d-e209-48ac-b474-52951a20fbc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:13.991716 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e33514d-e209-48ac-b474-52951a20fbc4 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:14.006513 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 363/741] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:14.006513 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33514d-e209-48ac-b474-52951a20fbc4 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:14.019971 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 379/742] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:13 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 00:08:14.021334 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78394302-fd4d-42e9-835a-2be362a92894 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:14.060228 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:14.068457 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:14.112964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33514d-e209-48ac-b474-52951a20fbc4 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:14.149233 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ddcd12-202b-44a5-9ea0-4d11deefe747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:14.149932 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddcd12-202b-44a5-9ea0-4d11deefe747 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:14.150308 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddcd12-202b-44a5-9ea0-4d11deefe747 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:14.150669 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddcd12-202b-44a5-9ea0-4d11deefe747 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:14.152465 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78394302-fd4d-42e9-835a-2be362a92894 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:14.162785 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ddcd12-202b-44a5-9ea0-4d11deefe747 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:14.164395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ddcd12-202b-44a5-9ea0-4d11deefe747 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:14.179742 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-c0ddcd12-202b-44a5-9ea0-4d11deefe747 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:14.179742 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:14.179742 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:14.179742 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:14.179742 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:14.179742 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:14.179742 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:14.179742 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:14.179742 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:14.179742 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:14.179742 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:14.179742 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:14.179742 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:14.179742 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:14.179742 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:14.179742 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:14.180907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:14.180907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:14.180907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:14.180907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:14.180907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:14.180907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:14.180907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:14.180907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:14.180907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:14.180907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:14.180907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:14.180907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:14.180907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:14.180907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:14.180907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:14.180907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:14.180907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:14.181840 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:14.181840 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:14.181840 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:14.182501 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 364/743] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:14.183561 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ddcd12-202b-44a5-9ea0-4d11deefe747 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:14.197480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24632a-5a4c-42c0-9049-01a331c29786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:14.198705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24632a-5a4c-42c0-9049-01a331c29786 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:14.199198 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24632a-5a4c-42c0-9049-01a331c29786 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:14.199876 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24632a-5a4c-42c0-9049-01a331c29786 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:14.201132 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec24632a-5a4c-42c0-9049-01a331c29786 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:14.201785 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec24632a-5a4c-42c0-9049-01a331c29786 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:14.203699 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec24632a-5a4c-42c0-9049-01a331c29786 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:14.214137 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 380/744] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:08:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:14.216245 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24632a-5a4c-42c0-9049-01a331c29786 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:14.313166 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ddcd12-202b-44a5-9ea0-4d11deefe747 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:14.316162 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:14.322885 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:14.327932 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24632a-5a4c-42c0-9049-01a331c29786 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:14.358524 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a9b03-0d8b-4829-8bf2-2402252a5052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:14.359486 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a9b03-0d8b-4829-8bf2-2402252a5052 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:14.359843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a9b03-0d8b-4829-8bf2-2402252a5052 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:14.360114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a9b03-0d8b-4829-8bf2-2402252a5052 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:14.365665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4269799-cbd4-4b50-adbf-1a1b55002680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:14.366273 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4269799-cbd4-4b50-adbf-1a1b55002680 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:14.366550 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4269799-cbd4-4b50-adbf-1a1b55002680 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:14.367089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4269799-cbd4-4b50-adbf-1a1b55002680 None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:14.367672 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4269799-cbd4-4b50-adbf-1a1b55002680 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:14.367784 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a9b03-0d8b-4829-8bf2-2402252a5052 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:14.367976 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4269799-cbd4-4b50-adbf-1a1b55002680 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:14.369129 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a9b03-0d8b-4829-8bf2-2402252a5052 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:14.369328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4269799-cbd4-4b50-adbf-1a1b55002680 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:14.381024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-333a9b03-0d8b-4829-8bf2-2402252a5052 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 00:08:14.381024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:14.381024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:14.381024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:14.381024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:14.381024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:14.381024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:14.381024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:14.381024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:14.381024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:14.381024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:14.381024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:14.381024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:08:14.381024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:08:14.381024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:08:14.381024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:08:14.381967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:14.381967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:14.381967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:08:14.381967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:14.381967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:08:14.381967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:14.381967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:14.381967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:14.381967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:14.381967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:14.381967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:14.381967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:14.381967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:14.381967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:14.381967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:14.381967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:14.381967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:14.381967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:14.383523 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:14.383523 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:14.383523 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:08:14.383523 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:08:14.383523 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:08:14.383523 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:08:14.383523 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:08:14.383523 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:08:14.383523 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 00:08:14.383523 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:14.383523 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 365/745] 10.222.0.67 () {56 vars in 1117 bytes} [Wed Aug 31 00:08:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:14.383523 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a9b03-0d8b-4829-8bf2-2402252a5052 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:14.387290 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 381/746] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:14.390100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4269799-cbd4-4b50-adbf-1a1b55002680 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:14.494241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a9b03-0d8b-4829-8bf2-2402252a5052 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:14.521075 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4269799-cbd4-4b50-adbf-1a1b55002680 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:14.544922 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38deb51-752c-4722-8195-7c480b871624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:14.545716 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38deb51-752c-4722-8195-7c480b871624 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:14.545996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38deb51-752c-4722-8195-7c480b871624 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:14.546264 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38deb51-752c-4722-8195-7c480b871624 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:14.558071 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38deb51-752c-4722-8195-7c480b871624 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:14.560122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38deb51-752c-4722-8195-7c480b871624 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:14.565810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c6cad-cf1f-447a-890f-1fe906ac51b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:14.566418 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c6cad-cf1f-447a-890f-1fe906ac51b5 None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:14.566693 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c6cad-cf1f-447a-890f-1fe906ac51b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:14.566962 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c6cad-cf1f-447a-890f-1fe906ac51b5 None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:14.568129 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c6cad-cf1f-447a-890f-1fe906ac51b5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:14.569979 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c6cad-cf1f-447a-890f-1fe906ac51b5 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:14.575866 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-d38deb51-752c-4722-8195-7c480b871624 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:14.575866 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:14.575866 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:14.575866 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:14.575866 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:14.575866 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:14.575866 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:14.575866 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:14.575866 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:14.575866 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:14.575866 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:14.575866 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:14.575866 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:14.575866 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:14.575866 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:14.575866 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:14.576909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:14.576909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:14.576909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:14.576909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:14.576909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:14.576909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:14.576909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:14.576909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:14.576909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:14.576909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:14.576909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:14.576909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:14.576909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:14.576909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:14.576909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:14.576909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:14.576909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:14.577900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:14.577900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:14.577900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:14.581841 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 366/747] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:14.582788 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38deb51-752c-4722-8195-7c480b871624 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:14.595077 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 382/748] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 31 00:08:14 2022] POST /identity/v3/services => generated 240 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:08:14.595077 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c6cad-cf1f-447a-890f-1fe906ac51b5 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:14.604474 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:14.635334 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:14.730718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38deb51-752c-4722-8195-7c480b871624 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:14.760579 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c6cad-cf1f-447a-890f-1fe906ac51b5 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:14.769217 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5001fb5-b5ae-4f3b-9c9a-2c1fde8c1e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:14.769825 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5001fb5-b5ae-4f3b-9c9a-2c1fde8c1e11 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:14.770106 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5001fb5-b5ae-4f3b-9c9a-2c1fde8c1e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:14.770376 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5001fb5-b5ae-4f3b-9c9a-2c1fde8c1e11 None admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:14.771133 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5001fb5-b5ae-4f3b-9c9a-2c1fde8c1e11 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:14.771447 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5001fb5-b5ae-4f3b-9c9a-2c1fde8c1e11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:14.772952 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5001fb5-b5ae-4f3b-9c9a-2c1fde8c1e11 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:14.798371 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 367/749] 10.222.0.67 () {56 vars in 1149 bytes} [Wed Aug 31 00:08:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:14.799667 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5001fb5-b5ae-4f3b-9c9a-2c1fde8c1e11 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:14.810478 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29105bd9-731a-4cc7-ad19-022965c4e248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:14.811144 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29105bd9-731a-4cc7-ad19-022965c4e248 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:14.811426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29105bd9-731a-4cc7-ad19-022965c4e248 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:14.811695 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29105bd9-731a-4cc7-ad19-022965c4e248 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:14.818710 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29105bd9-731a-4cc7-ad19-022965c4e248 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:14.820206 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29105bd9-731a-4cc7-ad19-022965c4e248 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:14.836944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-29105bd9-731a-4cc7-ad19-022965c4e248 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:14.836944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:14.836944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:14.836944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:14.836944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:14.836944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:14.836944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:14.836944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:14.836944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:14.836944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:14.836944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:14.836944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:14.836944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:08:14.836944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:08:14.836944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:08:14.837920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:08:14.837920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:14.837920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:14.837920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:14.837920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:14.837920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:14.837920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:14.837920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:14.837920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:14.837920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:14.837920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:14.837920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:14.837920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:14.837920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:14.837920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:14.837920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:14.837920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:14.838809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:08:14.838809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:08:14.838809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:08:14.838809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:08:14.838809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:08:14.838809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:08:14.838809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:14.838809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:14.840358 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 383/750] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:14.841709 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29105bd9-731a-4cc7-ad19-022965c4e248 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:14.900490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:14.934475 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:14.938044 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5001fb5-b5ae-4f3b-9c9a-2c1fde8c1e11 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:14.975556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29105bd9-731a-4cc7-ad19-022965c4e248 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:14.991493 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c499d74-f312-497e-94e5-b8276df83e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:14.992266 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c499d74-f312-497e-94e5-b8276df83e2d None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:14.992654 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c499d74-f312-497e-94e5-b8276df83e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:14.993102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c499d74-f312-497e-94e5-b8276df83e2d None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:14.993953 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c499d74-f312-497e-94e5-b8276df83e2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:14.994389 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c499d74-f312-497e-94e5-b8276df83e2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:14.996186 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c499d74-f312-497e-94e5-b8276df83e2d None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:15.015629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 368/751] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:15.016447 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c499d74-f312-497e-94e5-b8276df83e2d None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:15.034678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4556b-7bc7-4bd8-9a7b-5355a9cf8411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:15.034950 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4556b-7bc7-4bd8-9a7b-5355a9cf8411 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:15.039594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4556b-7bc7-4bd8-9a7b-5355a9cf8411 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:15.039594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4556b-7bc7-4bd8-9a7b-5355a9cf8411 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:15.128130 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4556b-7bc7-4bd8-9a7b-5355a9cf8411 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:15.129762 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4556b-7bc7-4bd8-9a7b-5355a9cf8411 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:15.158895 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c499d74-f312-497e-94e5-b8276df83e2d None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:15.205536 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cfdb94-cee7-4ccd-971d-23aaefce0e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:15.206777 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfdb94-cee7-4ccd-971d-23aaefce0e50 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:15.207581 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfdb94-cee7-4ccd-971d-23aaefce0e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:15.208983 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfdb94-cee7-4ccd-971d-23aaefce0e50 None admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:15.209899 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cfdb94-cee7-4ccd-971d-23aaefce0e50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:15.210414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cfdb94-cee7-4ccd-971d-23aaefce0e50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:15.212101 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cfdb94-cee7-4ccd-971d-23aaefce0e50 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:15.213983 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:15.215001 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:15.236179 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 369/752] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:08:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:15.236179 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cfdb94-cee7-4ccd-971d-23aaefce0e50 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:15.253009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 384/753] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:08:14 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:08:15.254236 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4556b-7bc7-4bd8-9a7b-5355a9cf8411 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:15.353961 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cfdb94-cee7-4ccd-971d-23aaefce0e50 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:15.374511 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4556b-7bc7-4bd8-9a7b-5355a9cf8411 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:15.393899 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976924f1-5f23-4d24-8da9-8970b8b029b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:15.394463 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976924f1-5f23-4d24-8da9-8970b8b029b1 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:15.394740 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976924f1-5f23-4d24-8da9-8970b8b029b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:15.395003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976924f1-5f23-4d24-8da9-8970b8b029b1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:15.402972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976924f1-5f23-4d24-8da9-8970b8b029b1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:15.404737 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976924f1-5f23-4d24-8da9-8970b8b029b1 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:15.412524 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24407d51-8af8-4da2-a934-33c6ebbcf8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:15.413329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24407d51-8af8-4da2-a934-33c6ebbcf8c7 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:15.413762 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24407d51-8af8-4da2-a934-33c6ebbcf8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:15.414207 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24407d51-8af8-4da2-a934-33c6ebbcf8c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:15.415226 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24407d51-8af8-4da2-a934-33c6ebbcf8c7 None admin] RBAC: Adding query filter params (role.id=34df06ce84ad4cf8b3bd1a4490228017, scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb, user.id=1cd3cfbc8ebd4c5a83bef09efe10bc91) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:15.415684 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24407d51-8af8-4da2-a934-33c6ebbcf8c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:15.416805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-976924f1-5f23-4d24-8da9-8970b8b029b1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:08:15.416805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:15.416805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:15.416805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:15.416805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:15.416805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:15.416805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:15.416805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:15.416805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:15.416805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:15.416805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:15.416805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:15.416805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:08:15.416805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:08:15.416805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:08:15.416805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:08:15.417928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:15.417928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:15.417928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:08:15.417928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:15.417928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:08:15.417928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:15.417928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:15.417928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:15.417928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:15.417928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:15.417928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:15.417928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:15.417928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:15.417928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:15.417928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:15.417928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:15.417928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:15.417928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:15.419781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:15.419781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:15.419781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:08:15.419781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:08:15.419781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:08:15.419781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:08:15.419781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:08:15.419781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:08:15.419781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:08:15.419781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:15.421024 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24407d51-8af8-4da2-a934-33c6ebbcf8c7 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:15.423231 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 370/754] 10.222.0.67 () {56 vars in 1093 bytes} [Wed Aug 31 00:08:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:15.424058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976924f1-5f23-4d24-8da9-8970b8b029b1 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:15.435191 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 385/755] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:08:15 2022] GET /identity/v3/role_assignments?user.id=1cd3cfbc8ebd4c5a83bef09efe10bc91&scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb&role.id=34df06ce84ad4cf8b3bd1a4490228017 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:15.436462 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24407d51-8af8-4da2-a934-33c6ebbcf8c7 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:15.488809 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:15.505146 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:15.543796 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976924f1-5f23-4d24-8da9-8970b8b029b1 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:15.574927 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24407d51-8af8-4da2-a934-33c6ebbcf8c7 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:15.583276 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af3899-0722-4768-a3d0-bedd69864f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:15.584178 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af3899-0722-4768-a3d0-bedd69864f38 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:15.584631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af3899-0722-4768-a3d0-bedd69864f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:15.585230 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af3899-0722-4768-a3d0-bedd69864f38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:15.598361 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af3899-0722-4768-a3d0-bedd69864f38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:15.600100 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af3899-0722-4768-a3d0-bedd69864f38 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:15.613086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31a5a3-73e9-4cb2-984d-7a9884c71a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:15.613762 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31a5a3-73e9-4cb2-984d-7a9884c71a82 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:15.614117 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31a5a3-73e9-4cb2-984d-7a9884c71a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:15.614461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31a5a3-73e9-4cb2-984d-7a9884c71a82 None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:15.615306 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31a5a3-73e9-4cb2-984d-7a9884c71a82 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:15.615688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31a5a3-73e9-4cb2-984d-7a9884c71a82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:15.617262 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31a5a3-73e9-4cb2-984d-7a9884c71a82 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:15.617908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-b0af3899-0722-4768-a3d0-bedd69864f38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:15.617908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:15.617908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:15.617908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:15.617908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:15.617908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:15.617908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:15.617908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:15.617908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:15.617908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:15.617908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:15.617908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:15.617908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:15.617908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:15.617908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:15.617908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:15.619003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:15.619003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:15.619003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:15.619003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:15.619003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:15.619003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:15.619003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:15.619003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:15.619003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:15.619003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:15.619003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:15.619003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:15.619003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:15.619003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:15.619003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:15.619003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:15.619003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:15.620110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:15.620110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:15.620110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:15.627074 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 371/756] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:15.627074 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18105b-30f3-4487-8786-a0027203edbf None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:15.627074 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18105b-30f3-4487-8786-a0027203edbf None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:15.627074 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18105b-30f3-4487-8786-a0027203edbf None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:15.630757 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 372/757] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:15.632331 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18105b-30f3-4487-8786-a0027203edbf None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:15.650678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 386/758] 10.222.0.67 () {56 vars in 1133 bytes} [Wed Aug 31 00:08:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:15.652708 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166a29d-284f-4591-95b7-992b198b083f None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:15.652708 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166a29d-284f-4591-95b7-992b198b083f None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:15.652878 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166a29d-284f-4591-95b7-992b198b083f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:15.675441 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-b166a29d-284f-4591-95b7-992b198b083f None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:15.677191 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b166a29d-284f-4591-95b7-992b198b083f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:15.763552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18105b-30f3-4487-8786-a0027203edbf None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:15.767194 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 387/759] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:15.768264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166a29d-284f-4591-95b7-992b198b083f None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:15.795512 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:15.807718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63a47e-3adc-442b-ae86-abfff703e444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:15.808508 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63a47e-3adc-442b-ae86-abfff703e444 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:15.808855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63a47e-3adc-442b-ae86-abfff703e444 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:15.809157 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63a47e-3adc-442b-ae86-abfff703e444 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:15.810579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a63a47e-3adc-442b-ae86-abfff703e444 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:15.811016 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a63a47e-3adc-442b-ae86-abfff703e444 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:15.812757 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a63a47e-3adc-442b-ae86-abfff703e444 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:15.829413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 373/760] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:15.840222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63a47e-3adc-442b-ae86-abfff703e444 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:15.842166 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:15.988868 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166a29d-284f-4591-95b7-992b198b083f None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:16.035288 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63a47e-3adc-442b-ae86-abfff703e444 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:16.062539 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c695a-a7a4-4233-a66a-8f22f8d65714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:16.063210 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c695a-a7a4-4233-a66a-8f22f8d65714 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:16.063489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c695a-a7a4-4233-a66a-8f22f8d65714 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:16.063759 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c695a-a7a4-4233-a66a-8f22f8d65714 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:16.082095 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c695a-a7a4-4233-a66a-8f22f8d65714 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:16.084217 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c695a-a7a4-4233-a66a-8f22f8d65714 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:16.090347 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0de448-3ac3-4b5e-96cb-16300f279739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:16.091073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0de448-3ac3-4b5e-96cb-16300f279739 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:16.091342 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0de448-3ac3-4b5e-96cb-16300f279739 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:16.091611 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0de448-3ac3-4b5e-96cb-16300f279739 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:16.105095 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0de448-3ac3-4b5e-96cb-16300f279739 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:16.107391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0de448-3ac3-4b5e-96cb-16300f279739 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:16.112910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-152c695a-a7a4-4233-a66a-8f22f8d65714 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:16.112910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:16.112910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:16.112910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:16.112910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:16.112910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:16.112910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:16.112910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:16.112910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:16.112910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:16.112910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:16.112910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:16.112910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:16.112910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:16.112910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:16.112910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:16.113835 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:16.113835 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:16.113835 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:16.113835 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:16.113835 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:16.113835 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:16.113835 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:16.113835 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:16.113835 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:16.113835 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:16.113835 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:16.113835 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:16.113835 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:16.113835 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:16.113835 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:16.113835 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:16.113835 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:16.114713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:16.114713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:16.114713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:16.114713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 388/761] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:16.115361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c695a-a7a4-4233-a66a-8f22f8d65714 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:16.128350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-2f0de448-3ac3-4b5e-96cb-16300f279739 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:16.128350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:16.128350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:16.128350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:16.128350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:16.128350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:16.128350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:16.128350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:16.128350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:16.128350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:16.128350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:16.128350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:16.128350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:08:16.128350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:08:16.128350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:08:16.129470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:08:16.129470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:16.129470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:16.129470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:16.129470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:16.129470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:16.129470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:16.129470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:16.129470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:16.129470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:16.129470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:16.129470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:16.129470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:16.129470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:16.129470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:16.129470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:16.129470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:16.130354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:08:16.130354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:08:16.130354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:08:16.130354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:08:16.130354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:08:16.130354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:08:16.130354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:16.130354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:16.130354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 374/762] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:16.130354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0de448-3ac3-4b5e-96cb-16300f279739 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:16.161763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:16.214199 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:16.238824 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c695a-a7a4-4233-a66a-8f22f8d65714 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:16.258890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0de448-3ac3-4b5e-96cb-16300f279739 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:16.276810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ad634-f38a-45a9-8f0c-bd0f6c0f7c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:16.277719 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ad634-f38a-45a9-8f0c-bd0f6c0f7c61 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:16.278136 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ad634-f38a-45a9-8f0c-bd0f6c0f7c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:16.278524 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ad634-f38a-45a9-8f0c-bd0f6c0f7c61 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:16.279324 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ad634-f38a-45a9-8f0c-bd0f6c0f7c61 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:16.280790 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ad634-f38a-45a9-8f0c-bd0f6c0f7c61 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:16.291426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-318ad634-f38a-45a9-8f0c-bd0f6c0f7c61 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 00:08:16.291426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:16.291426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:08:16.291426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:08:16.291426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:08:16.291426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:08:16.291426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:08:16.291426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:16.291426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 00:08:16.291426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:16.291426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:08:16.291426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:16.291426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:16.291426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:16.291426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:16.291426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:16.291426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:16.292415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:16.292415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:16.292415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:16.292415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:16.292415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:16.292415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:16.292415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:08:16.292415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:08:16.292415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:08:16.292415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:08:16.292415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:16.292415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:16.292415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:16.292415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:16.292415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:16.292415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:16.293248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:16.293248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:16.293248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:16.293248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:16.293248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:16.293248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:16.293248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:16.293248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:16.293248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:16.293248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:16.293248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:08:16.293248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:16.293248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 00:08:16.293248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:16.293248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 389/763] 10.222.0.67 () {56 vars in 1129 bytes} [Wed Aug 31 00:08:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:16.294325 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ad634-f38a-45a9-8f0c-bd0f6c0f7c61 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:16.306215 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740becd-f2a1-477e-a220-890ec32fa8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:16.307284 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740becd-f2a1-477e-a220-890ec32fa8ae None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:16.307762 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740becd-f2a1-477e-a220-890ec32fa8ae None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:16.308175 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740becd-f2a1-477e-a220-890ec32fa8ae None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:16.309147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a740becd-f2a1-477e-a220-890ec32fa8ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:16.310302 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a740becd-f2a1-477e-a220-890ec32fa8ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:16.311942 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a740becd-f2a1-477e-a220-890ec32fa8ae None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:16.330552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 375/764] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:16.331793 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740becd-f2a1-477e-a220-890ec32fa8ae None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:16.409137 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ad634-f38a-45a9-8f0c-bd0f6c0f7c61 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:16.426325 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:16.444136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740becd-f2a1-477e-a220-890ec32fa8ae None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:16.447757 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb76723-a5ed-4773-b6e3-4dbbcf292d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:16.448373 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb76723-a5ed-4773-b6e3-4dbbcf292d96 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:16.448653 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb76723-a5ed-4773-b6e3-4dbbcf292d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:16.448925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb76723-a5ed-4773-b6e3-4dbbcf292d96 None admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:16.451737 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb76723-a5ed-4773-b6e3-4dbbcf292d96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:16.452299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb76723-a5ed-4773-b6e3-4dbbcf292d96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:16.453947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb76723-a5ed-4773-b6e3-4dbbcf292d96 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:16.470745 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 390/765] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:08:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:16.472438 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb76723-a5ed-4773-b6e3-4dbbcf292d96 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:16.475340 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:16.494637 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed37c2-123c-412f-8687-6188a83a69a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:16.495281 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed37c2-123c-412f-8687-6188a83a69a5 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:16.495559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed37c2-123c-412f-8687-6188a83a69a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:16.495828 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed37c2-123c-412f-8687-6188a83a69a5 None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:16.496459 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bed37c2-123c-412f-8687-6188a83a69a5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:16.496766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bed37c2-123c-412f-8687-6188a83a69a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:16.498632 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bed37c2-123c-412f-8687-6188a83a69a5 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:16.509082 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 376/766] 10.222.0.67 () {56 vars in 1124 bytes} [Wed Aug 31 00:08:16 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:16.509082 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed37c2-123c-412f-8687-6188a83a69a5 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:16.602208 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb76723-a5ed-4773-b6e3-4dbbcf292d96 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:16.634831 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed37c2-123c-412f-8687-6188a83a69a5 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:16.642116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87533b-695f-404e-9f18-da00a564c3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:16.643625 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87533b-695f-404e-9f18-da00a564c3f3 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:16.643911 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87533b-695f-404e-9f18-da00a564c3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:16.644187 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87533b-695f-404e-9f18-da00a564c3f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:16.651733 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87533b-695f-404e-9f18-da00a564c3f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:16.653883 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87533b-695f-404e-9f18-da00a564c3f3 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:16.667344 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-ca87533b-695f-404e-9f18-da00a564c3f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:16.667344 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:16.667344 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:16.667344 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:16.667344 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:16.667344 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:16.667344 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:16.667344 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:16.667344 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:16.667344 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:16.667344 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:16.667344 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:16.667344 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:08:16.667344 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:08:16.667344 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:08:16.668329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:08:16.668329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:16.668329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:16.668329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:16.668329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:16.668329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:16.668329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:16.668329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:16.668329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:16.668329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:16.668329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:16.668329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:16.668329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:16.668329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:16.668329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:16.668329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:16.668329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:16.669426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:08:16.669426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:08:16.669426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:08:16.669426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:08:16.669426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:08:16.669426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:08:16.669426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:16.669426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:16.673593 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 391/767] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:16.674643 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87533b-695f-404e-9f18-da00a564c3f3 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:16.683136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6b4cc-6803-4927-8384-3a78c47dfdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:16.684737 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6b4cc-6803-4927-8384-3a78c47dfdf7 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:16.685293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6b4cc-6803-4927-8384-3a78c47dfdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:16.685776 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6b4cc-6803-4927-8384-3a78c47dfdf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:16.686737 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6b4cc-6803-4927-8384-3a78c47dfdf7 None admin] RBAC: Adding query filter params (role.id=4680aa64da80470b87075090fca2354b, scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb, user.id=a8fb7a9acefd4b418c992661037c2ed9) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:16.687278 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6b4cc-6803-4927-8384-3a78c47dfdf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:16.689828 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6b4cc-6803-4927-8384-3a78c47dfdf7 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:16.698034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 377/768] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:08:16 2022] GET /identity/v3/role_assignments?user.id=a8fb7a9acefd4b418c992661037c2ed9&scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb&role.id=4680aa64da80470b87075090fca2354b => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:16.699094 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6b4cc-6803-4927-8384-3a78c47dfdf7 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:16.719368 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:16.796036 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:16.828352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6b4cc-6803-4927-8384-3a78c47dfdf7 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:16.841514 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87533b-695f-404e-9f18-da00a564c3f3 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:16.866524 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d4614-e55f-45ce-ada3-07d8c71ec8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:16.867537 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d4614-e55f-45ce-ada3-07d8c71ec8fa None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:16.868098 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d4614-e55f-45ce-ada3-07d8c71ec8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:16.868525 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d4614-e55f-45ce-ada3-07d8c71ec8fa None admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:16.869382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d4614-e55f-45ce-ada3-07d8c71ec8fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:16.869854 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d4614-e55f-45ce-ada3-07d8c71ec8fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:16.871729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d4614-e55f-45ce-ada3-07d8c71ec8fa None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:16.880615 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d17b8f-3069-437f-9ae9-bf7ded131b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:16.881334 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d17b8f-3069-437f-9ae9-bf7ded131b0d None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:16.881721 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d17b8f-3069-437f-9ae9-bf7ded131b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:16.882058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d17b8f-3069-437f-9ae9-bf7ded131b0d None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:16.882769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d17b8f-3069-437f-9ae9-bf7ded131b0d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:16.883174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d17b8f-3069-437f-9ae9-bf7ded131b0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:16.885250 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d17b8f-3069-437f-9ae9-bf7ded131b0d None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:16.889148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 378/769] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:08:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:16.890775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0c251-06ce-497e-ad18-2bd81192e590 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:16.891269 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0c251-06ce-497e-ad18-2bd81192e590 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:16.891755 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0c251-06ce-497e-ad18-2bd81192e590 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:16.892604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 392/770] 10.222.0.67 () {56 vars in 1124 bytes} [Wed Aug 31 00:08:16 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:16.893356 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 379/771] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:16.898707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d17b8f-3069-437f-9ae9-bf7ded131b0d None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:16.899786 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec3335-1b21-4a88-847c-3a5a96ae3ad3 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:16.900289 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec3335-1b21-4a88-847c-3a5a96ae3ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:16.900746 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec3335-1b21-4a88-847c-3a5a96ae3ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:16.923315 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-bcec3335-1b21-4a88-847c-3a5a96ae3ad3 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:16.924901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-bcec3335-1b21-4a88-847c-3a5a96ae3ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:17.027193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 380/772] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:17.028647 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec3335-1b21-4a88-847c-3a5a96ae3ad3 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:17.034849 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:17.051258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d17b8f-3069-437f-9ae9-bf7ded131b0d None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:17.108890 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:17.132685 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf56e3-d304-47d9-a1aa-50e5c7a79d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:17.133365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf56e3-d304-47d9-a1aa-50e5c7a79d52 None admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:17.133647 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf56e3-d304-47d9-a1aa-50e5c7a79d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:17.133921 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf56e3-d304-47d9-a1aa-50e5c7a79d52 None admin] PATH_INFO: `/v3/projects/cd1e6daca4264a0cbc11a9160d1213eb/users/3aa53ae5a30146f69d7d6eefe4290dba/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:17.135930 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bf56e3-d304-47d9-a1aa-50e5c7a79d52 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd1e6daca4264a0cbc11a9160d1213eb, user_id=3aa53ae5a30146f69d7d6eefe4290dba, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:17.137324 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bf56e3-d304-47d9-a1aa-50e5c7a79d52 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:17.163455 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 393/773] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 31 00:08:16 2022] PUT /identity/v3/projects/cd1e6daca4264a0cbc11a9160d1213eb/users/3aa53ae5a30146f69d7d6eefe4290dba/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:08:17.191509 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec3335-1b21-4a88-847c-3a5a96ae3ad3 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:17.251205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2eab8a-c6e5-4e88-a54c-900b9da9d8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:17.251907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2eab8a-c6e5-4e88-a54c-900b9da9d8c8 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:17.252193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2eab8a-c6e5-4e88-a54c-900b9da9d8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:17.252465 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2eab8a-c6e5-4e88-a54c-900b9da9d8c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:17.253192 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2eab8a-c6e5-4e88-a54c-900b9da9d8c8 None admin] RBAC: Adding query filter params (interface=public, service_id=00a61da06914433e86703f00ee52df3a, region_id=RegionOne) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:17.253499 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2eab8a-c6e5-4e88-a54c-900b9da9d8c8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:17.254966 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2eab8a-c6e5-4e88-a54c-900b9da9d8c8 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:17.265884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 381/774] 10.222.0.67 () {56 vars in 1251 bytes} [Wed Aug 31 00:08:17 2022] GET /identity/v3/endpoints?service_id=00a61da06914433e86703f00ee52df3a&interface=public®ion_id=RegionOne => generated 194 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:17.272886 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf56e3-d304-47d9-a1aa-50e5c7a79d52 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:17.387934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2eab8a-c6e5-4e88-a54c-900b9da9d8c8 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:17.405889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf56e3-d304-47d9-a1aa-50e5c7a79d52 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:17.450650 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ef502-5135-40b0-93c2-5c2fe6dd94af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:17.451945 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ef502-5135-40b0-93c2-5c2fe6dd94af None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:17.452463 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ef502-5135-40b0-93c2-5c2fe6dd94af None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:17.452942 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ef502-5135-40b0-93c2-5c2fe6dd94af None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:17.453758 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ef502-5135-40b0-93c2-5c2fe6dd94af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:17.455430 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ef502-5135-40b0-93c2-5c2fe6dd94af None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:17.461539 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 394/775] 10.222.0.67 () {56 vars in 1087 bytes} [Wed Aug 31 00:08:17 2022] GET /identity/v3/services => generated 772 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:17.477609 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:17.517423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2eab8a-c6e5-4e88-a54c-900b9da9d8c8 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:17.579634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ed72a-7b9e-4dd0-8a0a-2f9266a20bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:17.580700 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ed72a-7b9e-4dd0-8a0a-2f9266a20bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:17.581192 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ed72a-7b9e-4dd0-8a0a-2f9266a20bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:17.581694 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ed72a-7b9e-4dd0-8a0a-2f9266a20bf7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:17.582505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ed72a-7b9e-4dd0-8a0a-2f9266a20bf7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:17.584219 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ed72a-7b9e-4dd0-8a0a-2f9266a20bf7 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:17.595683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-d33ed72a-7b9e-4dd0-8a0a-2f9266a20bf7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 00:08:17.595683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:17.595683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:08:17.595683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:08:17.595683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:08:17.595683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:08:17.595683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:08:17.595683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:17.595683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 00:08:17.595683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:17.595683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:08:17.595683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:17.595683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:17.595683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:17.595683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:17.595683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:17.595683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:17.597196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:17.597196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:17.597196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:17.597196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:17.597196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:17.597196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:17.597196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:08:17.597196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:08:17.597196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:08:17.597196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:08:17.597196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:17.597196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:17.597196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:17.597196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:17.597196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:17.597196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:17.598070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:17.598070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:17.598070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:17.598070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:17.598070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:17.598070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:17.598070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:17.598070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:17.598070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:17.598070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:17.598070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:08:17.598070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:17.598070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 00:08:17.598070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:17.601184 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 382/776] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:17 2022] GET /identity/v3/services/network => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:17.608569 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ef502-5135-40b0-93c2-5c2fe6dd94af None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:17.639773 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6334f0f-463b-468b-9650-5d7c9225256e None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:17.640081 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6334f0f-463b-468b-9650-5d7c9225256e None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:17.640263 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6334f0f-463b-468b-9650-5d7c9225256e None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:17.642059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 383/777] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:17.649630 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34fef9-dbb1-4297-95d5-c55a7e9a080a None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:17.649950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34fef9-dbb1-4297-95d5-c55a7e9a080a None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:17.650185 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34fef9-dbb1-4297-95d5-c55a7e9a080a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:17.672019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-0e34fef9-dbb1-4297-95d5-c55a7e9a080a None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:17.673508 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-0e34fef9-dbb1-4297-95d5-c55a7e9a080a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:17.694945 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:17.757858 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ef502-5135-40b0-93c2-5c2fe6dd94af None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:17.764281 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 384/778] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:08:17.799648 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6364f0-521e-428e-be74-d9a6926bbd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:17.800656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6364f0-521e-428e-be74-d9a6926bbd49 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:17.801145 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6364f0-521e-428e-be74-d9a6926bbd49 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:17.801623 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6364f0-521e-428e-be74-d9a6926bbd49 None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:17.802515 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6364f0-521e-428e-be74-d9a6926bbd49 None admin] RBAC: Adding query filter params (name=network) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:17.803024 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6364f0-521e-428e-be74-d9a6926bbd49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:17.804642 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6364f0-521e-428e-be74-d9a6926bbd49 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:17.810829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 395/779] 10.222.0.67 () {56 vars in 1112 bytes} [Wed Aug 31 00:08:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:17.816079 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e34fef9-dbb1-4297-95d5-c55a7e9a080a None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:17.884409 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:17.926018 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e34fef9-dbb1-4297-95d5-c55a7e9a080a None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:17.961030 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5b116-4038-4be0-90f0-12d4b12f2336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:17.961655 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5b116-4038-4be0-90f0-12d4b12f2336 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:17.961939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5b116-4038-4be0-90f0-12d4b12f2336 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:17.962214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5b116-4038-4be0-90f0-12d4b12f2336 None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:17.962861 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5b116-4038-4be0-90f0-12d4b12f2336 None admin] RBAC: Adding query filter params (type=network) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:17.963196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5b116-4038-4be0-90f0-12d4b12f2336 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:17.964582 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5b116-4038-4be0-90f0-12d4b12f2336 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:17.975837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 385/780] 10.222.0.67 () {56 vars in 1112 bytes} [Wed Aug 31 00:08:17 2022] GET /identity/v3/services?type=network => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:17.984372 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6364f0-521e-428e-be74-d9a6926bbd49 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:18.098598 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6364f0-521e-428e-be74-d9a6926bbd49 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:18.108126 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5b116-4038-4be0-90f0-12d4b12f2336 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:18.116036 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:18.140201 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd935ef-b668-4fef-9540-57def0de438a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:18.140824 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd935ef-b668-4fef-9540-57def0de438a None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:18.141109 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd935ef-b668-4fef-9540-57def0de438a None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:18.141390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd935ef-b668-4fef-9540-57def0de438a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:18.142100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd935ef-b668-4fef-9540-57def0de438a None admin] RBAC: Adding query filter params (interface=public, service_id=a7e7417efcdf4f37a85e75e976d5313d, region_id=RegionOne) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:18.142414 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd935ef-b668-4fef-9540-57def0de438a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:18.143850 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd935ef-b668-4fef-9540-57def0de438a None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:18.152143 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 396/781] 10.222.0.67 () {56 vars in 1251 bytes} [Wed Aug 31 00:08:17 2022] GET /identity/v3/endpoints?service_id=a7e7417efcdf4f37a85e75e976d5313d&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:18.160223 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd935ef-b668-4fef-9540-57def0de438a None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:18.269551 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5b116-4038-4be0-90f0-12d4b12f2336 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:18.301506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:18.314420 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf52d9-3cef-4168-b32b-424ce1b3b219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:18.315078 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf52d9-3cef-4168-b32b-424ce1b3b219 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:18.315368 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf52d9-3cef-4168-b32b-424ce1b3b219 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:18.315641 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf52d9-3cef-4168-b32b-424ce1b3b219 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:18.327470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf52d9-3cef-4168-b32b-424ce1b3b219 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:08:18.333407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-8aaf52d9-3cef-4168-b32b-424ce1b3b219 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:08:18.333407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:18.333407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:18.333407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:18.333407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:18.333407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:18.333407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:18.333407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:18.333407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:18.333407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:18.333407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:18.333407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:18.333407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:08:18.333407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:08:18.333407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:08:18.333407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:08:18.334344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:08:18.334344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:08:18.334344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:08:18.334344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:08:18.334344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:18.334344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:18.334344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:18.334344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:18.334344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:18.334344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:18.334344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:18.334344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:18.334344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:18.334344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:18.334344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:18.334344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:18.335352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:18.335352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:18.335352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:18.335352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:18.335352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:08:18.335352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:08:18.335352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:08:18.335352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:08:18.335352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:08:18.335352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:08:18.335352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:08:18.335352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:18.335352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 386/782] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:08:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:18.342359 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf52d9-3cef-4168-b32b-424ce1b3b219 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:18.374969 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd935ef-b668-4fef-9540-57def0de438a None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:18.447939 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb6649-cc2b-4833-8e1f-0efa5e0a9e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:18.448565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb6649-cc2b-4833-8e1f-0efa5e0a9e2c None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:18.448844 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb6649-cc2b-4833-8e1f-0efa5e0a9e2c None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:18.449120 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb6649-cc2b-4833-8e1f-0efa5e0a9e2c None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:18.449738 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb6649-cc2b-4833-8e1f-0efa5e0a9e2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:18.453246 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb6649-cc2b-4833-8e1f-0efa5e0a9e2c None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:18.464546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 397/783] 10.222.0.67 () {56 vars in 1087 bytes} [Wed Aug 31 00:08:18 2022] GET /identity/v3/services => generated 772 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:18.527626 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf52d9-3cef-4168-b32b-424ce1b3b219 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:18.571267 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0484141-14d7-46ae-af9e-7f8162a391d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:18.571900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0484141-14d7-46ae-af9e-7f8162a391d6 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:18.572328 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0484141-14d7-46ae-af9e-7f8162a391d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:18.572809 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0484141-14d7-46ae-af9e-7f8162a391d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:18.573503 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0484141-14d7-46ae-af9e-7f8162a391d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:18.573884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0484141-14d7-46ae-af9e-7f8162a391d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:18.575365 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0484141-14d7-46ae-af9e-7f8162a391d6 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:18.586323 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 387/784] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:08:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:18.593511 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb6649-cc2b-4833-8e1f-0efa5e0a9e2c None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:18.675025 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:18.736303 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb6649-cc2b-4833-8e1f-0efa5e0a9e2c None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:18.771921 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd7271-8ee5-4100-b522-5a98f41a4d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:18.772557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd7271-8ee5-4100-b522-5a98f41a4d26 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:18.772840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd7271-8ee5-4100-b522-5a98f41a4d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:18.773117 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd7271-8ee5-4100-b522-5a98f41a4d26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:18.784910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd7271-8ee5-4100-b522-5a98f41a4d26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:18.786370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd7271-8ee5-4100-b522-5a98f41a4d26 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:18.801810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-79dd7271-8ee5-4100-b522-5a98f41a4d26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:18.801810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:18.801810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:18.801810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:18.801810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:18.801810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:18.801810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:18.801810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:18.801810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:18.801810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:18.801810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:18.801810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:18.801810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:18.801810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:18.801810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:18.801810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:18.802793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:18.802793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:18.802793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:18.802793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:18.802793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:18.802793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:18.802793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:18.802793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:18.802793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:18.802793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:18.802793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:18.802793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:18.802793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:18.802793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:18.802793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:18.802793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:18.802793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:18.803989 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:18.803989 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:18.803989 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:18.803989 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 398/785] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:18.810908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0484141-14d7-46ae-af9e-7f8162a391d6 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:18.893748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:18.965295 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0484141-14d7-46ae-af9e-7f8162a391d6 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:19.030028 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412705a2-8340-48a8-a583-9c41ded602da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:19.030651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412705a2-8340-48a8-a583-9c41ded602da None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:19.030932 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412705a2-8340-48a8-a583-9c41ded602da None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:19.031235 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412705a2-8340-48a8-a583-9c41ded602da None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:19.031886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412705a2-8340-48a8-a583-9c41ded602da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:19.032191 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412705a2-8340-48a8-a583-9c41ded602da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:19.033553 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412705a2-8340-48a8-a583-9c41ded602da None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:19.056479 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 388/786] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:19.062193 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd7271-8ee5-4100-b522-5a98f41a4d26 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:19.167241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe797015-7a57-408f-a264-8ba6de92f1a1 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:19.167713 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe797015-7a57-408f-a264-8ba6de92f1a1 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:19.168093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe797015-7a57-408f-a264-8ba6de92f1a1 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:19.171242 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 389/787] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:19.176603 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12815924-64fd-4722-8d94-2a55c8c1a2ff None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:19.177001 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12815924-64fd-4722-8d94-2a55c8c1a2ff None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:19.177629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12815924-64fd-4722-8d94-2a55c8c1a2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:19.214647 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-12815924-64fd-4722-8d94-2a55c8c1a2ff None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:19.218510 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-12815924-64fd-4722-8d94-2a55c8c1a2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:19.240603 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:19.253979 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd7271-8ee5-4100-b522-5a98f41a4d26 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:19.315625 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20ac7c-e5dc-42a2-9d51-87637b96e17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:19.316636 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20ac7c-e5dc-42a2-9d51-87637b96e17c None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:19.317120 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20ac7c-e5dc-42a2-9d51-87637b96e17c None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:19.317721 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20ac7c-e5dc-42a2-9d51-87637b96e17c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:19.326145 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20ac7c-e5dc-42a2-9d51-87637b96e17c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:19.327974 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20ac7c-e5dc-42a2-9d51-87637b96e17c None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:19.340046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-6c20ac7c-e5dc-42a2-9d51-87637b96e17c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 00:08:19.340046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:19.340046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:19.340046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:19.340046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:19.340046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:19.340046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:19.340046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:19.340046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:19.340046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:19.340046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:19.340046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:19.340046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:08:19.340046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:08:19.340046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:08:19.340046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:08:19.341707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:19.341707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:19.341707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:08:19.341707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:19.341707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:08:19.341707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:19.341707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:19.341707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:19.341707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:19.341707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:19.341707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:19.341707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:19.341707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:19.341707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:19.341707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:19.341707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:19.341707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:19.341707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:19.342654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:19.342654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:19.342654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:08:19.342654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:08:19.342654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:08:19.342654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:08:19.342654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:08:19.342654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:08:19.342654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 00:08:19.342654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:19.349589 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 399/788] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:08:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:19.351127 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20ac7c-e5dc-42a2-9d51-87637b96e17c None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:19.373990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 390/789] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:19.458152 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20ac7c-e5dc-42a2-9d51-87637b96e17c None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:19.495573 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce5087-e70b-4b36-8f7d-c1f0c59eafd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:19.496536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce5087-e70b-4b36-8f7d-c1f0c59eafd5 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:19.497557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce5087-e70b-4b36-8f7d-c1f0c59eafd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:19.498049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce5087-e70b-4b36-8f7d-c1f0c59eafd5 None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:19.498987 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce5087-e70b-4b36-8f7d-c1f0c59eafd5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:19.499518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce5087-e70b-4b36-8f7d-c1f0c59eafd5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:19.501412 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce5087-e70b-4b36-8f7d-c1f0c59eafd5 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:19.525054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 400/790] 10.222.0.67 () {56 vars in 1135 bytes} [Wed Aug 31 00:08:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:19.526831 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12815924-64fd-4722-8d94-2a55c8c1a2ff None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:19.565870 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea32c37-ecca-4ed8-9690-5da4f2d7eb6c None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:19.566150 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea32c37-ecca-4ed8-9690-5da4f2d7eb6c None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:19.566353 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea32c37-ecca-4ed8-9690-5da4f2d7eb6c None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:19.567974 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 401/791] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:19.573073 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e02f57-6ee5-465c-ae26-c4d95378f017 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:19.573362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e02f57-6ee5-465c-ae26-c4d95378f017 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:19.573643 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e02f57-6ee5-465c-ae26-c4d95378f017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:19.587681 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:19.607663 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-a4e02f57-6ee5-465c-ae26-c4d95378f017 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:19.609358 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a4e02f57-6ee5-465c-ae26-c4d95378f017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:19.617928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:19.686210 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 402/792] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:08:19.706901 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e02f57-6ee5-465c-ae26-c4d95378f017 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:19.727295 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12815924-64fd-4722-8d94-2a55c8c1a2ff None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:19.764910 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243769b9-32e4-401c-8848-7745f3c7d18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:19.766058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243769b9-32e4-401c-8848-7745f3c7d18a None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:19.766541 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243769b9-32e4-401c-8848-7745f3c7d18a None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:19.767082 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243769b9-32e4-401c-8848-7745f3c7d18a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:19.779020 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243769b9-32e4-401c-8848-7745f3c7d18a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:19.780962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243769b9-32e4-401c-8848-7745f3c7d18a None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:19.806974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-243769b9-32e4-401c-8848-7745f3c7d18a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:19.806974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:19.806974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:19.806974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:19.806974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:19.806974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:19.806974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:19.806974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:19.806974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:19.806974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:19.806974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:19.806974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:19.806974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:19.806974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:19.806974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:19.806974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:19.808067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:19.808067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:19.808067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:19.808067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:19.808067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:19.808067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:19.808067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:19.808067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:19.808067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:19.808067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:19.808067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:19.808067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:19.808067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:19.808067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:19.808067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:19.808067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:19.808067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:19.808955 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:19.808955 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:19.808955 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:19.811064 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 391/793] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:19.817020 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243769b9-32e4-401c-8848-7745f3c7d18a None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:19.833767 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e02f57-6ee5-465c-ae26-c4d95378f017 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:19.876743 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a30142-9155-4d3e-a73c-8f2ef0f84fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:19.877380 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a30142-9155-4d3e-a73c-8f2ef0f84fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:19.877656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a30142-9155-4d3e-a73c-8f2ef0f84fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:19.877924 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a30142-9155-4d3e-a73c-8f2ef0f84fb4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:19.897632 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a30142-9155-4d3e-a73c-8f2ef0f84fb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:08:19.902536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-98a30142-9155-4d3e-a73c-8f2ef0f84fb4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:08:19.902536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:19.902536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:19.902536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:19.902536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:19.902536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:19.902536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:19.902536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:19.902536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:19.902536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:19.902536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:19.902536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:19.902536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:08:19.902536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:08:19.902536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:08:19.902536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:08:19.903769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:08:19.903769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:08:19.903769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:08:19.903769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:08:19.903769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:19.903769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:19.903769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:19.903769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:19.903769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:19.903769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:19.903769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:19.903769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:19.903769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:19.903769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:19.903769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:19.903769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:19.905067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:19.905067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:19.905067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:19.905067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:19.905067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:08:19.905067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:08:19.905067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:08:19.905067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:08:19.905067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:08:19.905067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:08:19.905067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:08:19.905067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:19.905067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 403/794] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 31 00:08:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:19.910246 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a30142-9155-4d3e-a73c-8f2ef0f84fb4 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:19.924218 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:20.041624 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a30142-9155-4d3e-a73c-8f2ef0f84fb4 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:20.046709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:20.056030 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243769b9-32e4-401c-8848-7745f3c7d18a None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:20.091629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff8c47-0728-42aa-aa9d-755db1260f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:20.092243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff8c47-0728-42aa-aa9d-755db1260f8f None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:20.092521 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff8c47-0728-42aa-aa9d-755db1260f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:20.092793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff8c47-0728-42aa-aa9d-755db1260f8f None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:20.093464 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff8c47-0728-42aa-aa9d-755db1260f8f None admin] RBAC: Adding query filter params (name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:20.093768 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff8c47-0728-42aa-aa9d-755db1260f8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:20.094959 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3488a-2adf-4c2c-922d-61595877d110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:20.095181 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff8c47-0728-42aa-aa9d-755db1260f8f None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:20.095604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3488a-2adf-4c2c-922d-61595877d110 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:20.095881 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3488a-2adf-4c2c-922d-61595877d110 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:20.096149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3488a-2adf-4c2c-922d-61595877d110 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:20.096973 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3488a-2adf-4c2c-922d-61595877d110 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:20.097611 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3488a-2adf-4c2c-922d-61595877d110 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:20.098968 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3488a-2adf-4c2c-922d-61595877d110 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:20.103645 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 404/795] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 31 00:08:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:20.105179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff8c47-0728-42aa-aa9d-755db1260f8f None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:20.116526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 392/796] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:20.117378 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3488a-2adf-4c2c-922d-61595877d110 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:20.195612 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:20.227081 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3488a-2adf-4c2c-922d-61595877d110 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:20.243304 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff8c47-0728-42aa-aa9d-755db1260f8f None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:20.270155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aabff2-26cb-406f-919b-fea4480b9cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:20.270789 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aabff2-26cb-406f-919b-fea4480b9cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:20.271091 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aabff2-26cb-406f-919b-fea4480b9cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:20.271368 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aabff2-26cb-406f-919b-fea4480b9cb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:20.279863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc829d-3bda-47ab-bef6-92629fdff235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:20.280694 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc829d-3bda-47ab-bef6-92629fdff235 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:20.280807 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc829d-3bda-47ab-bef6-92629fdff235 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:20.281039 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc829d-3bda-47ab-bef6-92629fdff235 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:20.282839 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aabff2-26cb-406f-919b-fea4480b9cb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:20.284316 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aabff2-26cb-406f-919b-fea4480b9cb8 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:20.291943 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdc829d-3bda-47ab-bef6-92629fdff235 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:08:20.298706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-2cdc829d-3bda-47ab-bef6-92629fdff235 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:08:20.298706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:20.298706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:20.298706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:20.298706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:20.298706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:20.298706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:20.298706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:20.298706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:20.298706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:20.298706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:20.298706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:20.298706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:08:20.298706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:08:20.298706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:08:20.298706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:08:20.299725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:08:20.299725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:08:20.299725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:08:20.299725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:08:20.299725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:20.299725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:20.299725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:20.299725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:20.299725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:20.299725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:20.299725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:20.299725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:20.299725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:20.299725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:20.299725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:20.299725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:20.301225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:20.301225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:20.301225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:20.301225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:20.301225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:08:20.301225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:08:20.301225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:08:20.301225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:08:20.301225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:08:20.301225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:08:20.301225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:08:20.301225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:20.301225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 405/797] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:08:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:20.301225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc829d-3bda-47ab-bef6-92629fdff235 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:20.415829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-87aabff2-26cb-406f-919b-fea4480b9cb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:20.415829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:20.415829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:20.415829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:20.415829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:20.415829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:20.415829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:20.415829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:20.415829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:20.415829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:20.415829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:20.415829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:20.415829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:20.415829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:20.415829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:20.415829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:20.417252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:20.417252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:20.417252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:20.417252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:20.417252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:20.417252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:20.417252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:20.417252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:20.417252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:20.417252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:20.417252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:20.417252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:20.417252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:20.417252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:20.417252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:20.417252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:20.417252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:20.418166 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:20.418166 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:20.418166 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:20.418166 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 393/798] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:20.418390 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d7004-9474-4330-a31c-a9f077e4950a None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:20.418594 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d7004-9474-4330-a31c-a9f077e4950a None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:20.418919 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d7004-9474-4330-a31c-a9f077e4950a None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:20.420335 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 394/799] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:20.420867 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d7004-9474-4330-a31c-a9f077e4950a None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:20.424906 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:20.427375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc829d-3bda-47ab-bef6-92629fdff235 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:20.452051 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:20.465091 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35224d38-b795-4059-9ed2-459ec3602b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:20.466486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35224d38-b795-4059-9ed2-459ec3602b2a None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:20.466913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35224d38-b795-4059-9ed2-459ec3602b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:20.467219 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35224d38-b795-4059-9ed2-459ec3602b2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:20.474646 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35224d38-b795-4059-9ed2-459ec3602b2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:20.476131 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35224d38-b795-4059-9ed2-459ec3602b2a None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:20.489328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-35224d38-b795-4059-9ed2-459ec3602b2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:20.489328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:20.489328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:20.489328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:20.489328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:20.489328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:20.489328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:20.489328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:20.489328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:20.489328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:20.489328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:20.489328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:20.489328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:08:20.489328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:08:20.489328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:08:20.490275 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:08:20.490275 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:20.490275 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:20.490275 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:20.490275 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:20.490275 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:20.490275 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:20.490275 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:20.490275 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:20.490275 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:20.490275 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:20.490275 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:20.490275 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:20.490275 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:20.490275 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:20.490275 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:20.490275 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:20.491670 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:08:20.491670 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:08:20.491670 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:08:20.491670 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:08:20.491670 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:08:20.491670 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:08:20.491670 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:20.491670 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:20.491670 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 406/800] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:20.491670 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35224d38-b795-4059-9ed2-459ec3602b2a None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:20.545621 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d7004-9474-4330-a31c-a9f077e4950a None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:20.587559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c0c4d-1690-42ac-85f8-5665c2a966bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:20.588193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c0c4d-1690-42ac-85f8-5665c2a966bb None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:20.588496 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c0c4d-1690-42ac-85f8-5665c2a966bb None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:20.588745 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c0c4d-1690-42ac-85f8-5665c2a966bb None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:20.589430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c0c4d-1690-42ac-85f8-5665c2a966bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:20.589733 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c0c4d-1690-42ac-85f8-5665c2a966bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:20.592124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c0c4d-1690-42ac-85f8-5665c2a966bb None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:20.607257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 395/801] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:08:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:20.607257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138ee6d-03c8-4f46-963c-b47d6656d1f6 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:20.607257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138ee6d-03c8-4f46-963c-b47d6656d1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:20.607257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138ee6d-03c8-4f46-963c-b47d6656d1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:20.623507 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-4138ee6d-03c8-4f46-963c-b47d6656d1f6 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:20.624732 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-4138ee6d-03c8-4f46-963c-b47d6656d1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:20.687656 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:20.698468 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 396/802] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:20.699875 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138ee6d-03c8-4f46-963c-b47d6656d1f6 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:20.727217 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35224d38-b795-4059-9ed2-459ec3602b2a None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:20.764909 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e95661-988d-4aa3-859e-ba19bec8a99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:20.765521 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e95661-988d-4aa3-859e-ba19bec8a99f None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:20.765802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e95661-988d-4aa3-859e-ba19bec8a99f None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:20.766079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e95661-988d-4aa3-859e-ba19bec8a99f None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:20.766734 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e95661-988d-4aa3-859e-ba19bec8a99f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:20.767019 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e95661-988d-4aa3-859e-ba19bec8a99f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:20.768423 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e95661-988d-4aa3-859e-ba19bec8a99f None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:20.784965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 407/803] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:20.784965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e95661-988d-4aa3-859e-ba19bec8a99f None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:20.822296 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138ee6d-03c8-4f46-963c-b47d6656d1f6 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:20.838429 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:20.862939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45469f0f-e50a-462a-85e0-18fe728d1161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:20.863593 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45469f0f-e50a-462a-85e0-18fe728d1161 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:20.863884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45469f0f-e50a-462a-85e0-18fe728d1161 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:20.864143 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45469f0f-e50a-462a-85e0-18fe728d1161 None admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:20.864831 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45469f0f-e50a-462a-85e0-18fe728d1161 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:20.865135 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45469f0f-e50a-462a-85e0-18fe728d1161 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:20.866553 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45469f0f-e50a-462a-85e0-18fe728d1161 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:20.883074 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 397/804] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:08:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:20.883074 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45469f0f-e50a-462a-85e0-18fe728d1161 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:20.906261 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e95661-988d-4aa3-859e-ba19bec8a99f None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:20.940174 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:20.945174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02604bc9-4f87-44ca-a586-25a7d28a6aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:20.945772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02604bc9-4f87-44ca-a586-25a7d28a6aea None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:20.946055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02604bc9-4f87-44ca-a586-25a7d28a6aea None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:20.946324 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02604bc9-4f87-44ca-a586-25a7d28a6aea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:20.958167 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02604bc9-4f87-44ca-a586-25a7d28a6aea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:20.959630 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02604bc9-4f87-44ca-a586-25a7d28a6aea None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:20.973760 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-02604bc9-4f87-44ca-a586-25a7d28a6aea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:20.973760 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:20.973760 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:20.973760 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:20.973760 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:20.973760 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:20.973760 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:20.973760 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:20.973760 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:20.973760 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:20.973760 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:20.973760 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:20.973760 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:20.973760 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:20.973760 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:20.973760 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:20.974750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:20.974750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:20.974750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:20.974750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:20.974750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:20.974750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:20.974750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:20.974750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:20.974750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:20.974750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:20.974750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:20.974750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:20.974750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:20.974750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:20.974750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:20.974750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:20.974750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:20.975654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:20.975654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:20.975654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:20.975654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 408/805] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:20.976321 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1837804-8093-4864-931e-8f2f52608664 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:20.976603 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1837804-8093-4864-931e-8f2f52608664 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:20.976871 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1837804-8093-4864-931e-8f2f52608664 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:20.978078 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 409/806] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:20.978744 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1837804-8093-4864-931e-8f2f52608664 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:20.998914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45469f0f-e50a-462a-85e0-18fe728d1161 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:21.032675 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c36614-2277-460b-b3bd-7e63c0e399d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:21.033300 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c36614-2277-460b-b3bd-7e63c0e399d6 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:21.033579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c36614-2277-460b-b3bd-7e63c0e399d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:21.033853 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c36614-2277-460b-b3bd-7e63c0e399d6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:21.041223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c36614-2277-460b-b3bd-7e63c0e399d6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:21.042659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c36614-2277-460b-b3bd-7e63c0e399d6 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:21.053651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-85c36614-2277-460b-b3bd-7e63c0e399d6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 00:08:21.053651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:21.053651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:21.053651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:21.053651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:21.053651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:21.053651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:21.053651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:21.053651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:21.053651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:21.053651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:21.053651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:21.053651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:08:21.053651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:08:21.053651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:08:21.053651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:08:21.054629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:21.054629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:21.054629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:08:21.054629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:21.054629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:08:21.054629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:21.054629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:21.054629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:21.054629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:21.054629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:21.054629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:21.054629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:21.054629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:21.054629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:21.054629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:21.054629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:21.054629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:21.054629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:21.055642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:21.055642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:21.055642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:08:21.055642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:08:21.055642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:08:21.055642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:08:21.055642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:08:21.055642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:08:21.055642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 00:08:21.055642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:21.055642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 398/807] 10.222.0.67 () {56 vars in 1117 bytes} [Wed Aug 31 00:08:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:21.055642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c36614-2277-460b-b3bd-7e63c0e399d6 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:21.083160 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:21.090333 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1837804-8093-4864-931e-8f2f52608664 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:21.125071 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f5f2b-21f5-4232-b17b-030d5a5d611d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:21.125985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f5f2b-21f5-4232-b17b-030d5a5d611d None admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:21.126270 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f5f2b-21f5-4232-b17b-030d5a5d611d None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:21.126545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f5f2b-21f5-4232-b17b-030d5a5d611d None admin] PATH_INFO: `/v3/projects/cd1e6daca4264a0cbc11a9160d1213eb/users/1cd3cfbc8ebd4c5a83bef09efe10bc91/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:21.128632 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f5f2b-21f5-4232-b17b-030d5a5d611d None admin] RBAC: Authorizing `identity:create_grant(project_id=cd1e6daca4264a0cbc11a9160d1213eb, user_id=1cd3cfbc8ebd4c5a83bef09efe10bc91, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:21.130060 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f5f2b-21f5-4232-b17b-030d5a5d611d None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:21.144265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 410/808] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 31 00:08:20 2022] PUT /identity/v3/projects/cd1e6daca4264a0cbc11a9160d1213eb/users/1cd3cfbc8ebd4c5a83bef09efe10bc91/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:08:21.145254 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f5f2b-21f5-4232-b17b-030d5a5d611d None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:21.162349 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c36614-2277-460b-b3bd-7e63c0e399d6 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:21.180218 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:21.205896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c61bfe7-9185-43e8-bebe-249dfea3ac6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:21.206453 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61bfe7-9185-43e8-bebe-249dfea3ac6f None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:21.206729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61bfe7-9185-43e8-bebe-249dfea3ac6f None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:21.207002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61bfe7-9185-43e8-bebe-249dfea3ac6f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:21.207627 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c61bfe7-9185-43e8-bebe-249dfea3ac6f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:21.209010 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c61bfe7-9185-43e8-bebe-249dfea3ac6f None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:21.218855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-3c61bfe7-9185-43e8-bebe-249dfea3ac6f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 00:08:21.218855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:21.218855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:08:21.218855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:08:21.218855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:08:21.218855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:08:21.218855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:08:21.218855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:21.218855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 00:08:21.218855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:21.218855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:08:21.218855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:21.218855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:21.218855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:21.218855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:21.218855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:21.218855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:21.220136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:21.220136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:21.220136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:21.220136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:21.220136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:21.220136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:21.220136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:08:21.220136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:08:21.220136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:08:21.220136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:08:21.220136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:21.220136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:21.220136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:21.220136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:21.220136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:21.220136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:21.220971 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:21.220971 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:21.220971 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:21.220971 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:21.220971 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:21.220971 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:21.220971 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:21.220971 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:21.220971 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:21.220971 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:21.220971 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:08:21.220971 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:21.220971 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 00:08:21.220971 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:21.220971 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 399/809] 10.222.0.67 () {56 vars in 1129 bytes} [Wed Aug 31 00:08:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:21.221847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea171de-afe7-4ee3-8cf3-b658bf9beb06 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:21.221934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea171de-afe7-4ee3-8cf3-b658bf9beb06 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:21.222136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea171de-afe7-4ee3-8cf3-b658bf9beb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:21.242068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-cea171de-afe7-4ee3-8cf3-b658bf9beb06 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:21.243307 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-cea171de-afe7-4ee3-8cf3-b658bf9beb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:21.281229 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f5f2b-21f5-4232-b17b-030d5a5d611d None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:21.320652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 400/810] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:21.321510 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea171de-afe7-4ee3-8cf3-b658bf9beb06 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:21.330671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c877ac-3afe-40b1-8fae-16bf0b829a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:21.331647 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c877ac-3afe-40b1-8fae-16bf0b829a90 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:21.332618 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c877ac-3afe-40b1-8fae-16bf0b829a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:21.333096 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c877ac-3afe-40b1-8fae-16bf0b829a90 None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:21.334359 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c877ac-3afe-40b1-8fae-16bf0b829a90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:21.335186 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c877ac-3afe-40b1-8fae-16bf0b829a90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:21.337630 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c877ac-3afe-40b1-8fae-16bf0b829a90 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:21.355315 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 411/811] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:21.356719 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c877ac-3afe-40b1-8fae-16bf0b829a90 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:21.379881 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:21.428385 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea171de-afe7-4ee3-8cf3-b658bf9beb06 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:21.440078 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:21.468079 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152c2a3-7fd1-42e5-b478-82ff9a928064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:21.468606 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152c2a3-7fd1-42e5-b478-82ff9a928064 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:21.468885 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152c2a3-7fd1-42e5-b478-82ff9a928064 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:21.469153 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152c2a3-7fd1-42e5-b478-82ff9a928064 None admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:21.469881 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7152c2a3-7fd1-42e5-b478-82ff9a928064 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:21.470182 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7152c2a3-7fd1-42e5-b478-82ff9a928064 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:21.471658 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7152c2a3-7fd1-42e5-b478-82ff9a928064 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:21.494218 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 401/812] 10.222.0.67 () {56 vars in 1149 bytes} [Wed Aug 31 00:08:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:21.495193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152c2a3-7fd1-42e5-b478-82ff9a928064 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:21.500524 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c877ac-3afe-40b1-8fae-16bf0b829a90 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:21.551247 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd46bc8-d5d7-4fa6-b301-25b0c92808f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:21.551948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd46bc8-d5d7-4fa6-b301-25b0c92808f1 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:21.552228 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd46bc8-d5d7-4fa6-b301-25b0c92808f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:21.552495 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd46bc8-d5d7-4fa6-b301-25b0c92808f1 None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:21.553119 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd46bc8-d5d7-4fa6-b301-25b0c92808f1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:21.553419 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd46bc8-d5d7-4fa6-b301-25b0c92808f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:21.554875 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd46bc8-d5d7-4fa6-b301-25b0c92808f1 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:21.564781 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 412/813] 10.222.0.67 () {56 vars in 1124 bytes} [Wed Aug 31 00:08:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:21.565765 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd46bc8-d5d7-4fa6-b301-25b0c92808f1 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:21.604392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152c2a3-7fd1-42e5-b478-82ff9a928064 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:21.641835 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2cbe47-2af6-40b2-9e3c-6c3b016a1bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:21.642468 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cbe47-2af6-40b2-9e3c-6c3b016a1bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:21.642745 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cbe47-2af6-40b2-9e3c-6c3b016a1bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:21.643015 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cbe47-2af6-40b2-9e3c-6c3b016a1bc1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:21.651146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2cbe47-2af6-40b2-9e3c-6c3b016a1bc1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:21.651740 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:21.654252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2cbe47-2af6-40b2-9e3c-6c3b016a1bc1 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:21.667821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-bd2cbe47-2af6-40b2-9e3c-6c3b016a1bc1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:08:21.667821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:21.667821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:21.667821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:21.667821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:21.667821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:21.667821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:21.667821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:21.667821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:21.667821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:21.667821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:21.667821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:21.667821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:08:21.667821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:08:21.667821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:08:21.667821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:08:21.668898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:21.668898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:21.668898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:08:21.668898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:21.668898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:08:21.668898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:21.668898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:21.668898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:21.668898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:21.668898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:21.668898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:21.668898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:21.668898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:21.668898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:21.668898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:21.668898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:21.668898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:21.668898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:21.669843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:21.669843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:21.669843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:08:21.669843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:08:21.669843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:08:21.669843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:08:21.669843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:08:21.669843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:08:21.669843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:08:21.669843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:21.669843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 402/814] 10.222.0.67 () {56 vars in 1093 bytes} [Wed Aug 31 00:08:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:21.670924 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2cbe47-2af6-40b2-9e3c-6c3b016a1bc1 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:21.679121 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd46bc8-d5d7-4fa6-b301-25b0c92808f1 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:21.701788 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:21.717228 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e2836-8076-4c81-a80c-7c0eed7ea40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:21.717859 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e2836-8076-4c81-a80c-7c0eed7ea40c None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:21.718184 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e2836-8076-4c81-a80c-7c0eed7ea40c None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:21.718399 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e2836-8076-4c81-a80c-7c0eed7ea40c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:21.729920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e2836-8076-4c81-a80c-7c0eed7ea40c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:21.731442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e2836-8076-4c81-a80c-7c0eed7ea40c None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:21.746058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-885e2836-8076-4c81-a80c-7c0eed7ea40c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:21.746058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:21.746058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:21.746058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:21.746058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:21.746058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:21.746058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:21.746058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:21.746058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:21.746058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:21.746058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:21.746058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:21.746058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:21.746058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:21.746058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:21.746058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:21.747656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:21.747656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:21.747656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:21.747656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:21.747656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:21.747656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:21.747656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:21.747656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:21.747656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:21.747656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:21.747656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:21.747656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:21.747656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:21.747656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:21.747656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:21.747656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:21.747656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:21.748733 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:21.748733 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:21.748733 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:21.748733 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 413/815] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:21.748733 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e2836-8076-4c81-a80c-7c0eed7ea40c None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:21.786878 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2cbe47-2af6-40b2-9e3c-6c3b016a1bc1 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:21.822887 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc343b-585e-4cc9-99e0-5c7b25d95f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:21.823789 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc343b-585e-4cc9-99e0-5c7b25d95f19 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:21.824231 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc343b-585e-4cc9-99e0-5c7b25d95f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:21.824557 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc343b-585e-4cc9-99e0-5c7b25d95f19 None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:21.826161 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc343b-585e-4cc9-99e0-5c7b25d95f19 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:21.826161 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc343b-585e-4cc9-99e0-5c7b25d95f19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:21.827516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc343b-585e-4cc9-99e0-5c7b25d95f19 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:21.832617 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 403/816] 10.222.0.67 () {56 vars in 1124 bytes} [Wed Aug 31 00:08:21 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:21.833406 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc343b-585e-4cc9-99e0-5c7b25d95f19 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:21.860376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e2836-8076-4c81-a80c-7c0eed7ea40c None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:21.893651 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:21.900068 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22498bc7-fc07-4344-ae68-349bf8f42378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:21.900687 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22498bc7-fc07-4344-ae68-349bf8f42378 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:21.901076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22498bc7-fc07-4344-ae68-349bf8f42378 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:21.901351 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22498bc7-fc07-4344-ae68-349bf8f42378 None admin] PATH_INFO: `/v3/services/network` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:21.902050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22498bc7-fc07-4344-ae68-349bf8f42378 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:21.903953 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22498bc7-fc07-4344-ae68-349bf8f42378 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:21.912993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-22498bc7-fc07-4344-ae68-349bf8f42378 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 00:08:21.912993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:21.912993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:08:21.912993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:08:21.912993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:08:21.912993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:08:21.912993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:08:21.912993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:21.912993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 00:08:21.912993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:21.912993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:08:21.912993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:21.912993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:21.912993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:21.912993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:21.912993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:21.912993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:21.913968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:21.913968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:21.913968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:21.913968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:21.913968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:21.913968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:21.913968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:08:21.913968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:08:21.913968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:08:21.913968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:08:21.913968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:21.913968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:21.913968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:21.913968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:21.913968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:21.913968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:21.914784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:21.914784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:21.914784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:21.914784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:21.914784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:21.914784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:21.914784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:21.914784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:21.914784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:21.914784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:21.914784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:08:21.914784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:21.914784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 00:08:21.914784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:21.919535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 414/817] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:21 2022] GET /identity/v3/services/network => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:21.920397 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22498bc7-fc07-4344-ae68-349bf8f42378 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:21.933911 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:21.943974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc343b-585e-4cc9-99e0-5c7b25d95f19 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:21.987434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162850a9-016f-49cf-9c1d-8339c33a61a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:21.987965 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162850a9-016f-49cf-9c1d-8339c33a61a1 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:21.988303 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162850a9-016f-49cf-9c1d-8339c33a61a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:21.988579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162850a9-016f-49cf-9c1d-8339c33a61a1 None admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:21.991883 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162850a9-016f-49cf-9c1d-8339c33a61a1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:21.991883 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162850a9-016f-49cf-9c1d-8339c33a61a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:21.991883 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162850a9-016f-49cf-9c1d-8339c33a61a1 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:22.010333 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 404/818] 10.222.0.67 () {56 vars in 1133 bytes} [Wed Aug 31 00:08:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:22.010961 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162850a9-016f-49cf-9c1d-8339c33a61a1 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:22.047346 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22498bc7-fc07-4344-ae68-349bf8f42378 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:22.083854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c486eb95-548b-49d8-ab46-aa2c6d340f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:22.084557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486eb95-548b-49d8-ab46-aa2c6d340f37 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:22.084908 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486eb95-548b-49d8-ab46-aa2c6d340f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:22.085259 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c486eb95-548b-49d8-ab46-aa2c6d340f37 None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:22.085968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c486eb95-548b-49d8-ab46-aa2c6d340f37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:22.086342 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c486eb95-548b-49d8-ab46-aa2c6d340f37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:22.087819 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c486eb95-548b-49d8-ab46-aa2c6d340f37 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:22.100926 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 415/819] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:22.104214 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c486eb95-548b-49d8-ab46-aa2c6d340f37 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:22.121040 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162850a9-016f-49cf-9c1d-8339c33a61a1 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:22.158709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de56fa0-15f2-410f-a3c0-64cdc5f3a8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:22.159344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de56fa0-15f2-410f-a3c0-64cdc5f3a8d2 None admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:22.159975 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de56fa0-15f2-410f-a3c0-64cdc5f3a8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:22.160254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de56fa0-15f2-410f-a3c0-64cdc5f3a8d2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:22.160908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de56fa0-15f2-410f-a3c0-64cdc5f3a8d2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:22.162569 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de56fa0-15f2-410f-a3c0-64cdc5f3a8d2 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:22.176106 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:22.185437 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 405/820] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:08:22 2022] POST /identity/v3/endpoints => generated 346 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:08:22.192135 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de56fa0-15f2-410f-a3c0-64cdc5f3a8d2 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:22.201850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:22.236152 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c486eb95-548b-49d8-ab46-aa2c6d340f37 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:22.288280 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846fb2b8-1fa3-4c8c-a38b-6d6330d8d24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:22.289006 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fb2b8-1fa3-4c8c-a38b-6d6330d8d24b None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:22.289327 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fb2b8-1fa3-4c8c-a38b-6d6330d8d24b None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:22.289599 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fb2b8-1fa3-4c8c-a38b-6d6330d8d24b None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:22.290240 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846fb2b8-1fa3-4c8c-a38b-6d6330d8d24b None admin] RBAC: Adding query filter params (name=network) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:22.290542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846fb2b8-1fa3-4c8c-a38b-6d6330d8d24b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:22.291997 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846fb2b8-1fa3-4c8c-a38b-6d6330d8d24b None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:22.297543 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 416/821] 10.222.0.67 () {56 vars in 1112 bytes} [Wed Aug 31 00:08:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:22.303376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846fb2b8-1fa3-4c8c-a38b-6d6330d8d24b None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:22.348831 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de56fa0-15f2-410f-a3c0-64cdc5f3a8d2 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:22.386679 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960ac07-32c1-480a-bd8b-ba7ec8e95c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:22.387101 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960ac07-32c1-480a-bd8b-ba7ec8e95c93 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:22.387394 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960ac07-32c1-480a-bd8b-ba7ec8e95c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:22.388360 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960ac07-32c1-480a-bd8b-ba7ec8e95c93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:22.434028 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846fb2b8-1fa3-4c8c-a38b-6d6330d8d24b None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:22.461458 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:22.474356 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960ac07-32c1-480a-bd8b-ba7ec8e95c93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:22.476485 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960ac07-32c1-480a-bd8b-ba7ec8e95c93 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:22.480197 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:22.488892 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152e85c-1656-4ae2-989f-0af6163bbfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:22.489581 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152e85c-1656-4ae2-989f-0af6163bbfef None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:22.489863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152e85c-1656-4ae2-989f-0af6163bbfef None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:22.490133 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152e85c-1656-4ae2-989f-0af6163bbfef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:22.500151 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c152e85c-1656-4ae2-989f-0af6163bbfef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:22.501954 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c152e85c-1656-4ae2-989f-0af6163bbfef None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:22.517138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-c152e85c-1656-4ae2-989f-0af6163bbfef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:22.517138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:22.517138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:22.517138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:22.517138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:22.517138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:22.517138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:22.517138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:22.517138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:22.517138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:22.517138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:22.517138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:22.517138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:08:22.517138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:08:22.517138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:08:22.518076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:08:22.518076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:22.518076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:22.518076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:22.518076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:22.518076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:22.518076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:22.518076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:22.518076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:22.518076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:22.518076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:22.518076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:22.518076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:22.518076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:22.518076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:22.518076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:22.518076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:22.518977 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:08:22.518977 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:08:22.518977 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:08:22.518977 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:08:22.518977 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:08:22.518977 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:08:22.518977 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:22.518977 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:22.521458 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 417/822] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:22.526903 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152e85c-1656-4ae2-989f-0af6163bbfef None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:22.615172 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 406/823] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:08:22 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:08:22.617199 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960ac07-32c1-480a-bd8b-ba7ec8e95c93 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:22.646000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152e85c-1656-4ae2-989f-0af6163bbfef None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:22.685628 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bac5a1-fc1b-4783-96d7-08fb031218ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:22.686623 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bac5a1-fc1b-4783-96d7-08fb031218ff None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:22.687157 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bac5a1-fc1b-4783-96d7-08fb031218ff None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:22.687631 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bac5a1-fc1b-4783-96d7-08fb031218ff None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:22.688457 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bac5a1-fc1b-4783-96d7-08fb031218ff None admin] RBAC: Adding query filter params (type=network) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:22.688957 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bac5a1-fc1b-4783-96d7-08fb031218ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:22.690536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bac5a1-fc1b-4783-96d7-08fb031218ff None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:22.701601 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 418/824] 10.222.0.67 () {56 vars in 1112 bytes} [Wed Aug 31 00:08:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:22.702982 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bac5a1-fc1b-4783-96d7-08fb031218ff None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:22.749277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960ac07-32c1-480a-bd8b-ba7ec8e95c93 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:22.753425 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:22.755225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:22.801774 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626b312-b7ad-4a31-a4b6-3daa1423f6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:22.802386 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626b312-b7ad-4a31-a4b6-3daa1423f6c4 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:22.802662 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626b312-b7ad-4a31-a4b6-3daa1423f6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:22.802941 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626b312-b7ad-4a31-a4b6-3daa1423f6c4 None admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:22.804118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c626b312-b7ad-4a31-a4b6-3daa1423f6c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:22.804428 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c626b312-b7ad-4a31-a4b6-3daa1423f6c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:22.806107 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c626b312-b7ad-4a31-a4b6-3daa1423f6c4 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:22.827073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 407/825] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:08:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:22.827073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626b312-b7ad-4a31-a4b6-3daa1423f6c4 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:22.828960 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bac5a1-fc1b-4783-96d7-08fb031218ff None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:22.875118 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7b799-f91a-4164-91b5-84ce901683e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:22.876381 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7b799-f91a-4164-91b5-84ce901683e9 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:22.876674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7b799-f91a-4164-91b5-84ce901683e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:22.876947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7b799-f91a-4164-91b5-84ce901683e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:22.891532 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b7b799-f91a-4164-91b5-84ce901683e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:22.893272 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b7b799-f91a-4164-91b5-84ce901683e9 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:22.911780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-59b7b799-f91a-4164-91b5-84ce901683e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:22.911780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:22.911780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:22.911780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:22.911780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:22.911780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:22.911780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:22.911780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:22.911780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:22.911780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:22.911780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:22.911780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:22.911780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:22.911780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:22.911780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:22.911780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:22.912797 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:22.912797 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:22.912797 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:22.912797 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:22.912797 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:22.912797 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:22.912797 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:22.912797 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:22.912797 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:22.912797 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:22.912797 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:22.912797 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:22.912797 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:22.912797 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:22.912797 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:22.912797 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:22.912797 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:22.913658 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:22.913658 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:22.913658 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:22.915810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 419/826] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:22.916884 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7b799-f91a-4164-91b5-84ce901683e9 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:22.956134 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626b312-b7ad-4a31-a4b6-3daa1423f6c4 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:22.998544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477d1ae-22b5-4f7f-886b-220551c8d333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:22.999499 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477d1ae-22b5-4f7f-886b-220551c8d333 None admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:23.000001 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477d1ae-22b5-4f7f-886b-220551c8d333 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:23.000471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477d1ae-22b5-4f7f-886b-220551c8d333 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:23.001586 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9477d1ae-22b5-4f7f-886b-220551c8d333 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:23.003301 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9477d1ae-22b5-4f7f-886b-220551c8d333 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:23.027736 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:23.039292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7b799-f91a-4164-91b5-84ce901683e9 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:23.040002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 408/827] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:08:22 2022] POST /identity/v3/endpoints => generated 337 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:08:23.041590 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477d1ae-22b5-4f7f-886b-220551c8d333 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:23.098755 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:23.109914 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f7be6-d9dd-4c56-b1e1-eb01774725ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:23.110602 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f7be6-d9dd-4c56-b1e1-eb01774725ea None admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:23.110883 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f7be6-d9dd-4c56-b1e1-eb01774725ea None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:23.111185 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f7be6-d9dd-4c56-b1e1-eb01774725ea None admin] PATH_INFO: `/v3/projects/cd1e6daca4264a0cbc11a9160d1213eb/users/a8fb7a9acefd4b418c992661037c2ed9/roles/4680aa64da80470b87075090fca2354b` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:23.133105 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f7be6-d9dd-4c56-b1e1-eb01774725ea None admin] RBAC: Authorizing `identity:create_grant(project_id=cd1e6daca4264a0cbc11a9160d1213eb, user_id=a8fb7a9acefd4b418c992661037c2ed9, role_id=4680aa64da80470b87075090fca2354b)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:23.134358 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f7be6-d9dd-4c56-b1e1-eb01774725ea None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:23.151429 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 420/828] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 31 00:08:22 2022] PUT /identity/v3/projects/cd1e6daca4264a0cbc11a9160d1213eb/users/a8fb7a9acefd4b418c992661037c2ed9/roles/4680aa64da80470b87075090fca2354b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:08:23.159834 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204749b7-e2aa-44e6-bcdf-5038d7a01f2e None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:23.160175 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204749b7-e2aa-44e6-bcdf-5038d7a01f2e None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:23.160507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204749b7-e2aa-44e6-bcdf-5038d7a01f2e None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:23.166682 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 421/829] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:23 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:23.168465 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b80168-bc61-4528-9b3f-441207ba3af4 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:23.168863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b80168-bc61-4528-9b3f-441207ba3af4 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:23.169746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b80168-bc61-4528-9b3f-441207ba3af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:23.195439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-04b80168-bc61-4528-9b3f-441207ba3af4 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:23.197005 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-04b80168-bc61-4528-9b3f-441207ba3af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:23.218085 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477d1ae-22b5-4f7f-886b-220551c8d333 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:23.259861 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d7f62-9a1c-41d6-a6b8-e3323caaf1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:23.260920 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d7f62-9a1c-41d6-a6b8-e3323caaf1e2 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:23.261458 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d7f62-9a1c-41d6-a6b8-e3323caaf1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:23.261932 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d7f62-9a1c-41d6-a6b8-e3323caaf1e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:23.263438 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d7f62-9a1c-41d6-a6b8-e3323caaf1e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:23.263993 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d7f62-9a1c-41d6-a6b8-e3323caaf1e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:23.265775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d7f62-9a1c-41d6-a6b8-e3323caaf1e2 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:23.280704 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 409/830] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:23.282994 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 422/831] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:23 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:08:23.286073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d7f62-9a1c-41d6-a6b8-e3323caaf1e2 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:23.380011 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:23.397919 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d7f62-9a1c-41d6-a6b8-e3323caaf1e2 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:23.438194 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bbb1fb-69b1-4cc7-939a-038f15e1a044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:23.438831 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbb1fb-69b1-4cc7-939a-038f15e1a044 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:23.439133 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbb1fb-69b1-4cc7-939a-038f15e1a044 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:23.439409 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbb1fb-69b1-4cc7-939a-038f15e1a044 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:23.446387 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bbb1fb-69b1-4cc7-939a-038f15e1a044 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:23.447845 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bbb1fb-69b1-4cc7-939a-038f15e1a044 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:23.461007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-e2bbb1fb-69b1-4cc7-939a-038f15e1a044 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:23.461007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:23.461007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:23.461007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:23.461007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:23.461007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:23.461007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:23.461007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:23.461007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:23.461007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:23.461007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:23.461007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:23.461007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:08:23.461007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:08:23.461007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:08:23.461935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:08:23.461935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:23.461935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:23.461935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:23.461935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:23.461935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:23.461935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:23.461935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:23.461935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:23.461935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:23.461935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:23.461935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:23.461935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:23.461935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:23.461935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:23.461935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:23.461935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:23.462811 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:08:23.462811 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:08:23.462811 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:08:23.462811 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:08:23.462811 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:08:23.462811 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:08:23.462811 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:23.462811 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:23.462811 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 410/832] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:23.468518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b80168-bc61-4528-9b3f-441207ba3af4 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:23.553945 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:23.583903 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bbb1fb-69b1-4cc7-939a-038f15e1a044 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:23.599240 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b80168-bc61-4528-9b3f-441207ba3af4 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:23.648660 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a209a8-1c2b-4110-ad09-80accfeecb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:23.649902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a209a8-1c2b-4110-ad09-80accfeecb94 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:23.650375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a209a8-1c2b-4110-ad09-80accfeecb94 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:23.651415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a209a8-1c2b-4110-ad09-80accfeecb94 None admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:23.652998 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a209a8-1c2b-4110-ad09-80accfeecb94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:23.654134 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a209a8-1c2b-4110-ad09-80accfeecb94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:23.655850 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a209a8-1c2b-4110-ad09-80accfeecb94 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:23.672907 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 423/833] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:08:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:23.679277 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a209a8-1c2b-4110-ad09-80accfeecb94 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:23.721773 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bbb1fb-69b1-4cc7-939a-038f15e1a044 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:23.766998 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896acf1-bc8e-470d-8d17-1665b130a29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:23.768121 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896acf1-bc8e-470d-8d17-1665b130a29e None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:23.769424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896acf1-bc8e-470d-8d17-1665b130a29e None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:23.769829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896acf1-bc8e-470d-8d17-1665b130a29e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:23.780278 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:23.784503 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3896acf1-bc8e-470d-8d17-1665b130a29e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:08:23.791285 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-3896acf1-bc8e-470d-8d17-1665b130a29e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:08:23.791285 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:23.791285 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:23.791285 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:23.791285 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:23.791285 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:23.791285 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:23.791285 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:23.791285 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:23.791285 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:23.791285 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:23.791285 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:23.791285 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:08:23.791285 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:08:23.791285 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:08:23.791285 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:08:23.792412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:08:23.792412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:08:23.792412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:08:23.792412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:08:23.792412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:23.792412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:23.792412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:23.792412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:23.792412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:23.792412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:23.792412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:23.792412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:23.792412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:23.792412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:23.792412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:23.792412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:23.793395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:23.793395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:23.793395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:23.793395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:23.793395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:08:23.793395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:08:23.793395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:08:23.793395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:08:23.793395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:08:23.793395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:08:23.793395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:08:23.793395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:23.796427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 411/834] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:08:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:23.801766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896acf1-bc8e-470d-8d17-1665b130a29e None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:23.816124 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a209a8-1c2b-4110-ad09-80accfeecb94 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:23.849437 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:23.860010 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148d34e-4e80-4e45-8f49-94fd37122501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:23.861025 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148d34e-4e80-4e45-8f49-94fd37122501 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:23.861311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148d34e-4e80-4e45-8f49-94fd37122501 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:23.861629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148d34e-4e80-4e45-8f49-94fd37122501 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:23.862356 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2148d34e-4e80-4e45-8f49-94fd37122501 None admin] RBAC: Adding query filter params (role.id=34df06ce84ad4cf8b3bd1a4490228017, scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb, user.id=3aa53ae5a30146f69d7d6eefe4290dba) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:23.862663 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2148d34e-4e80-4e45-8f49-94fd37122501 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:23.864740 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2148d34e-4e80-4e45-8f49-94fd37122501 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:23.879139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 424/835] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:08:23 2022] GET /identity/v3/role_assignments?user.id=3aa53ae5a30146f69d7d6eefe4290dba&scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb&role.id=34df06ce84ad4cf8b3bd1a4490228017 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:23.936964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896acf1-bc8e-470d-8d17-1665b130a29e None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:23.977101 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ad68c-0469-496a-8edf-28bab70f949f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:23.978299 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ad68c-0469-496a-8edf-28bab70f949f None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:23.978816 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ad68c-0469-496a-8edf-28bab70f949f None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:23.979341 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ad68c-0469-496a-8edf-28bab70f949f None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:23.980305 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326ad68c-0469-496a-8edf-28bab70f949f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:23.980816 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326ad68c-0469-496a-8edf-28bab70f949f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:23.983062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326ad68c-0469-496a-8edf-28bab70f949f None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:23.999067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 412/836] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:08:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:24.002530 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148d34e-4e80-4e45-8f49-94fd37122501 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:24.118483 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148d34e-4e80-4e45-8f49-94fd37122501 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:24.157616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3b31a-9a3c-4c70-90d9-34adf6e4418d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:24.158250 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3b31a-9a3c-4c70-90d9-34adf6e4418d None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:24.158524 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3b31a-9a3c-4c70-90d9-34adf6e4418d None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:24.158794 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3b31a-9a3c-4c70-90d9-34adf6e4418d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:24.170535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3b31a-9a3c-4c70-90d9-34adf6e4418d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:24.172018 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3b31a-9a3c-4c70-90d9-34adf6e4418d None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:24.187440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-dfa3b31a-9a3c-4c70-90d9-34adf6e4418d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:24.187440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:24.187440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:24.187440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:24.187440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:24.187440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:24.187440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:24.187440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:24.187440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:24.187440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:24.187440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:24.187440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:24.187440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:24.187440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:24.187440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:24.187440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:24.188365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:24.188365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:24.188365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:24.188365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:24.188365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:24.188365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:24.188365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:24.188365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:24.188365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:24.188365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:24.188365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:24.188365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:24.188365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:24.188365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:24.188365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:24.188365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:24.188365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:24.189232 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:24.189232 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:24.189232 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:24.189232 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 425/837] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:24.194678 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ad68c-0469-496a-8edf-28bab70f949f None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:24.264043 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:24.296172 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2c92b-1fe7-45b1-875a-c8d8aa90c195 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:24.296485 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2c92b-1fe7-45b1-875a-c8d8aa90c195 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:24.296727 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2c92b-1fe7-45b1-875a-c8d8aa90c195 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:24.298191 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 426/838] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:24.305141 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70bf1a-7547-44f6-ab08-0c85da52a44d None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:24.305455 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70bf1a-7547-44f6-ab08-0c85da52a44d None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:24.305645 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70bf1a-7547-44f6-ab08-0c85da52a44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:24.306323 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ad68c-0469-496a-8edf-28bab70f949f None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:24.324849 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-0e70bf1a-7547-44f6-ab08-0c85da52a44d None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:24.326123 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-0e70bf1a-7547-44f6-ab08-0c85da52a44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:24.346398 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356887c-7f18-4cab-a01f-24f9725091c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:24.347010 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356887c-7f18-4cab-a01f-24f9725091c8 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:24.347312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356887c-7f18-4cab-a01f-24f9725091c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:24.347582 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356887c-7f18-4cab-a01f-24f9725091c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:24.348749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4356887c-7f18-4cab-a01f-24f9725091c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:24.349106 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4356887c-7f18-4cab-a01f-24f9725091c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:24.350987 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4356887c-7f18-4cab-a01f-24f9725091c8 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:24.364785 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 413/839] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:24.370255 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356887c-7f18-4cab-a01f-24f9725091c8 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:24.370549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:24.407010 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 427/840] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:24.495693 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356887c-7f18-4cab-a01f-24f9725091c8 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:24.520068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:24.532947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69682006-113e-4d5b-906d-c6a3b565afb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:24.533889 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69682006-113e-4d5b-906d-c6a3b565afb8 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:24.534364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69682006-113e-4d5b-906d-c6a3b565afb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:24.534851 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69682006-113e-4d5b-906d-c6a3b565afb8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:24.542868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69682006-113e-4d5b-906d-c6a3b565afb8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:24.544624 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69682006-113e-4d5b-906d-c6a3b565afb8 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:24.556097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-69682006-113e-4d5b-906d-c6a3b565afb8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 00:08:24.556097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:24.556097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:24.556097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:24.556097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:24.556097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:24.556097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:24.556097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:24.556097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:24.556097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:24.556097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:24.556097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:24.556097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:08:24.556097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:08:24.556097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:08:24.556097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:08:24.557041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:24.557041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:24.557041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:08:24.557041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:24.557041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:08:24.557041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:24.557041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:24.557041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:24.557041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:24.557041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:24.557041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:24.557041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:24.557041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:24.557041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:24.557041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:24.557041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:24.557041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:24.557041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:24.558058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:24.558058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:24.558058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:08:24.558058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:08:24.558058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:08:24.558058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:08:24.558058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:08:24.558058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:08:24.558058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 00:08:24.558058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:24.561239 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 414/841] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:08:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:24.568159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70bf1a-7547-44f6-ab08-0c85da52a44d None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:24.681700 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69682006-113e-4d5b-906d-c6a3b565afb8 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:24.683133 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70bf1a-7547-44f6-ab08-0c85da52a44d None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:24.720321 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf39919-76e7-44da-954e-52317f80e497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:24.720983 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf39919-76e7-44da-954e-52317f80e497 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:24.721261 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf39919-76e7-44da-954e-52317f80e497 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:24.721528 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf39919-76e7-44da-954e-52317f80e497 None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:24.722416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf39919-76e7-44da-954e-52317f80e497 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:24.722722 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf39919-76e7-44da-954e-52317f80e497 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:24.724189 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf39919-76e7-44da-954e-52317f80e497 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:24.746407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 428/842] 10.222.0.67 () {56 vars in 1135 bytes} [Wed Aug 31 00:08:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:24.752288 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf39919-76e7-44da-954e-52317f80e497 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:24.785056 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:24.801075 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69682006-113e-4d5b-906d-c6a3b565afb8 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:24.835204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eea104-78bb-4b42-a701-fd6288a08b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:24.835972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eea104-78bb-4b42-a701-fd6288a08b66 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:24.836085 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eea104-78bb-4b42-a701-fd6288a08b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:24.836355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eea104-78bb-4b42-a701-fd6288a08b66 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:24.837083 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eea104-78bb-4b42-a701-fd6288a08b66 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:24.838448 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eea104-78bb-4b42-a701-fd6288a08b66 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:24.850350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-91eea104-78bb-4b42-a701-fd6288a08b66 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 00:08:24.850350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:24.850350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:08:24.850350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:08:24.850350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:08:24.850350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:08:24.850350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:08:24.850350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:24.850350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 00:08:24.850350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:24.850350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:08:24.850350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:24.850350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:24.850350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:24.850350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:24.850350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:24.850350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:24.851652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:24.851652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:24.851652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:24.851652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:24.851652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:24.851652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:24.851652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:08:24.851652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:08:24.851652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:08:24.851652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:08:24.851652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:24.851652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:24.851652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:24.851652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:24.851652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:24.851652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:24.852485 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:24.852485 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:24.852485 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:24.852485 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:24.852485 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:24.852485 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:24.852485 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:24.852485 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:24.852485 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:24.852485 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:24.852485 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:08:24.852485 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:24.852485 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 00:08:24.852485 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:24.852485 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 415/843] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 31 00:08:24 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:24.857259 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eea104-78bb-4b42-a701-fd6288a08b66 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:24.873920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf39919-76e7-44da-954e-52317f80e497 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:24.917394 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:24.923488 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c97cd9-dd2e-43f9-8b51-f98063b99a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:24.924471 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c97cd9-dd2e-43f9-8b51-f98063b99a38 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:24.924755 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c97cd9-dd2e-43f9-8b51-f98063b99a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:24.925021 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c97cd9-dd2e-43f9-8b51-f98063b99a38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:25.002097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eea104-78bb-4b42-a701-fd6288a08b66 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:25.014585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c97cd9-dd2e-43f9-8b51-f98063b99a38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:25.016168 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c97cd9-dd2e-43f9-8b51-f98063b99a38 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:25.047813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd5a63-74a8-45bd-a17b-a395ed8fe609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:25.049223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd5a63-74a8-45bd-a17b-a395ed8fe609 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:25.049844 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd5a63-74a8-45bd-a17b-a395ed8fe609 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:25.050432 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd5a63-74a8-45bd-a17b-a395ed8fe609 None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:25.051283 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd5a63-74a8-45bd-a17b-a395ed8fe609 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:25.051801 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd5a63-74a8-45bd-a17b-a395ed8fe609 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:25.054281 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd5a63-74a8-45bd-a17b-a395ed8fe609 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:25.065492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 416/844] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 31 00:08:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:25.065668 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:25.067101 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd5a63-74a8-45bd-a17b-a395ed8fe609 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:25.144295 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 429/845] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:08:24 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:08:25.149589 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c97cd9-dd2e-43f9-8b51-f98063b99a38 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:25.199809 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd5a63-74a8-45bd-a17b-a395ed8fe609 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:25.215660 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:25.245773 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae23bcf-54c7-40d5-9d50-d1f7ef673e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:25.246392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae23bcf-54c7-40d5-9d50-d1f7ef673e1f None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:25.246680 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae23bcf-54c7-40d5-9d50-d1f7ef673e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:25.247726 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae23bcf-54c7-40d5-9d50-d1f7ef673e1f None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:25.248491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae23bcf-54c7-40d5-9d50-d1f7ef673e1f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:25.248800 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae23bcf-54c7-40d5-9d50-d1f7ef673e1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:25.250163 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae23bcf-54c7-40d5-9d50-d1f7ef673e1f None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:25.255340 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 417/846] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 31 00:08:25 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:25.263212 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee21f17-5cd1-4516-bdee-9dd98d6a9a04 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:25.263212 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee21f17-5cd1-4516-bdee-9dd98d6a9a04 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:25.263212 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee21f17-5cd1-4516-bdee-9dd98d6a9a04 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:25.263212 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 418/847] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:25.265199 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f909e67-8c1d-4c0a-902b-803f708b646e None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:25.265570 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f909e67-8c1d-4c0a-902b-803f708b646e None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:25.265927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f909e67-8c1d-4c0a-902b-803f708b646e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:25.270432 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c97cd9-dd2e-43f9-8b51-f98063b99a38 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:25.286344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-3f909e67-8c1d-4c0a-902b-803f708b646e None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:25.287721 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-3f909e67-8c1d-4c0a-902b-803f708b646e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:25.307480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:25.313689 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3b71d-e9c2-45ef-a209-cf5af88dfefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:25.314335 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3b71d-e9c2-45ef-a209-cf5af88dfefa None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:25.314613 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3b71d-e9c2-45ef-a209-cf5af88dfefa None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:25.314884 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3b71d-e9c2-45ef-a209-cf5af88dfefa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:25.326531 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff3b71d-e9c2-45ef-a209-cf5af88dfefa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:25.328027 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff3b71d-e9c2-45ef-a209-cf5af88dfefa None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:25.345000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-8ff3b71d-e9c2-45ef-a209-cf5af88dfefa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:25.345000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:25.345000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:25.345000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:25.345000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:25.345000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:25.345000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:25.345000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:25.345000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:25.345000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:25.345000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:25.345000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:25.345000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:25.345000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:25.345000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:25.345000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:25.346045 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:25.346045 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:25.346045 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:25.346045 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:25.346045 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:25.346045 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:25.346045 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:25.346045 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:25.346045 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:25.346045 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:25.346045 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:25.346045 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:25.346045 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:25.346045 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:25.346045 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:25.346045 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:25.346045 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:25.346950 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:25.346950 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:25.346950 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:25.351496 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 430/848] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:25.354201 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3b71d-e9c2-45ef-a209-cf5af88dfefa None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:25.403121 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 419/849] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:25.476303 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3b71d-e9c2-45ef-a209-cf5af88dfefa None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:25.513158 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700b685-88da-4c63-9754-e1e7e27145a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:25.513698 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700b685-88da-4c63-9754-e1e7e27145a5 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:25.513979 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700b685-88da-4c63-9754-e1e7e27145a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:25.514258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700b685-88da-4c63-9754-e1e7e27145a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:25.514906 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700b685-88da-4c63-9754-e1e7e27145a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:25.515234 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700b685-88da-4c63-9754-e1e7e27145a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:25.516682 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700b685-88da-4c63-9754-e1e7e27145a5 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:25.529885 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 431/850] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:25.535216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f909e67-8c1d-4c0a-902b-803f708b646e None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:25.646465 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f909e67-8c1d-4c0a-902b-803f708b646e None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:25.648867 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:25.681871 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700b685-88da-4c63-9754-e1e7e27145a5 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:25.684884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed52dd-e774-4af0-b47e-9a8cf8a6c8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:25.685632 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed52dd-e774-4af0-b47e-9a8cf8a6c8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:25.685903 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed52dd-e774-4af0-b47e-9a8cf8a6c8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:25.686177 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed52dd-e774-4af0-b47e-9a8cf8a6c8e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:25.693420 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed52dd-e774-4af0-b47e-9a8cf8a6c8e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:25.694870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed52dd-e774-4af0-b47e-9a8cf8a6c8e6 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:25.707021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-55ed52dd-e774-4af0-b47e-9a8cf8a6c8e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:25.707021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:25.707021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:25.707021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:25.707021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:25.707021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:25.707021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:25.707021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:25.707021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:25.707021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:25.707021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:25.707021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:25.707021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:08:25.707021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:08:25.707021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:08:25.707981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:08:25.707981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:25.707981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:25.707981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:25.707981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:25.707981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:25.707981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:25.707981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:25.707981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:25.707981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:25.707981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:25.707981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:25.707981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:25.707981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:25.707981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:25.707981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:25.707981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:25.708879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:08:25.708879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:08:25.708879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:08:25.708879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:08:25.708879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:08:25.708879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:08:25.708879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:25.708879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:25.708879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 420/851] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:25.716818 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed52dd-e774-4af0-b47e-9a8cf8a6c8e6 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:25.736762 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:25.811673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700b685-88da-4c63-9754-e1e7e27145a5 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:25.847732 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed52dd-e774-4af0-b47e-9a8cf8a6c8e6 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:25.858980 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ceed0-09b0-4eff-aeb4-0974ae92cb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:25.859659 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ceed0-09b0-4eff-aeb4-0974ae92cb64 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:25.859935 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ceed0-09b0-4eff-aeb4-0974ae92cb64 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:25.860198 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ceed0-09b0-4eff-aeb4-0974ae92cb64 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:25.876432 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ceed0-09b0-4eff-aeb4-0974ae92cb64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:08:25.883329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-403ceed0-09b0-4eff-aeb4-0974ae92cb64 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:08:25.883329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:25.883329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:25.883329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:25.883329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:25.883329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:25.883329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:25.883329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:25.883329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:25.883329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:25.883329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:25.883329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:25.883329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:08:25.883329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:08:25.883329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:08:25.883329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:08:25.884260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:08:25.884260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:08:25.884260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:08:25.884260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:08:25.884260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:25.884260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:25.884260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:25.884260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:25.884260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:25.884260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:25.884260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:25.884260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:25.884260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:25.884260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:25.884260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:25.884260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:25.885104 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:25.885104 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:25.885104 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:25.885104 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:25.885104 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:08:25.885104 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:08:25.885104 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:08:25.885104 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:08:25.885104 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:08:25.885104 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:08:25.885104 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:08:25.885104 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:25.885104 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 432/852] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 31 00:08:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:25.893642 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ceed0-09b0-4eff-aeb4-0974ae92cb64 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:25.895551 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7587c-f71f-4ad3-a227-2c046d12e2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:25.896163 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7587c-f71f-4ad3-a227-2c046d12e2fa None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:25.896439 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7587c-f71f-4ad3-a227-2c046d12e2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:25.896709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7587c-f71f-4ad3-a227-2c046d12e2fa None admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:25.897458 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7587c-f71f-4ad3-a227-2c046d12e2fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:25.897765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7587c-f71f-4ad3-a227-2c046d12e2fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:25.900497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7587c-f71f-4ad3-a227-2c046d12e2fa None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:25.922595 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 421/853] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:08:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:25.924209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7587c-f71f-4ad3-a227-2c046d12e2fa None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:25.959223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:26.017347 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ceed0-09b0-4eff-aeb4-0974ae92cb64 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:26.021368 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:26.053330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7587c-f71f-4ad3-a227-2c046d12e2fa None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:26.055860 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea6e563-f3bc-40a8-b978-276fbf59a8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:26.056828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6e563-f3bc-40a8-b978-276fbf59a8c4 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:26.056828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6e563-f3bc-40a8-b978-276fbf59a8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:26.057001 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6e563-f3bc-40a8-b978-276fbf59a8c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:26.057611 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea6e563-f3bc-40a8-b978-276fbf59a8c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:26.057911 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea6e563-f3bc-40a8-b978-276fbf59a8c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:26.059288 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea6e563-f3bc-40a8-b978-276fbf59a8c4 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:26.068315 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 433/854] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 31 00:08:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:26.069465 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06d511-098c-4d24-9aed-305f31e125d3 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:26.069753 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06d511-098c-4d24-9aed-305f31e125d3 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:26.070025 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06d511-098c-4d24-9aed-305f31e125d3 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:26.075222 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 434/855] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:26.076546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06d511-098c-4d24-9aed-305f31e125d3 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:26.097696 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b3c31-9f3d-4093-a5d4-76aed79f7ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:26.098310 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b3c31-9f3d-4093-a5d4-76aed79f7ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:26.098620 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b3c31-9f3d-4093-a5d4-76aed79f7ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:26.099141 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b3c31-9f3d-4093-a5d4-76aed79f7ad6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:26.099721 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b3c31-9f3d-4093-a5d4-76aed79f7ad6 None admin] RBAC: Adding query filter params (role.id=34df06ce84ad4cf8b3bd1a4490228017, scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb, user.id=1cd3cfbc8ebd4c5a83bef09efe10bc91) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:26.100033 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b3c31-9f3d-4093-a5d4-76aed79f7ad6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:26.101476 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b3c31-9f3d-4093-a5d4-76aed79f7ad6 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:26.119118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 422/856] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:08:25 2022] GET /identity/v3/role_assignments?user.id=1cd3cfbc8ebd4c5a83bef09efe10bc91&scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb&role.id=34df06ce84ad4cf8b3bd1a4490228017 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:26.119118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0cdf4-9403-4736-9739-e91fc89f7981 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:26.119118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0cdf4-9403-4736-9739-e91fc89f7981 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:26.119118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0cdf4-9403-4736-9739-e91fc89f7981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:26.137219 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-4cf0cdf4-9403-4736-9739-e91fc89f7981 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:26.138774 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-4cf0cdf4-9403-4736-9739-e91fc89f7981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:26.212235 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06d511-098c-4d24-9aed-305f31e125d3 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:26.220449 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 423/857] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:26.252148 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74140d10-cfc0-46ef-92c6-00223ad4ad66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:26.252805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74140d10-cfc0-46ef-92c6-00223ad4ad66 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:26.253087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74140d10-cfc0-46ef-92c6-00223ad4ad66 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:26.253361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74140d10-cfc0-46ef-92c6-00223ad4ad66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:26.265933 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74140d10-cfc0-46ef-92c6-00223ad4ad66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:26.267429 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74140d10-cfc0-46ef-92c6-00223ad4ad66 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:26.283390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-74140d10-cfc0-46ef-92c6-00223ad4ad66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:26.283390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:26.283390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:26.283390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:26.283390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:26.283390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:26.283390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:26.283390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:26.283390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:26.283390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:26.283390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:26.283390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:26.283390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:26.283390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:26.283390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:26.283390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:26.284570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:26.284570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:26.284570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:26.284570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:26.284570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:26.284570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:26.284570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:26.284570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:26.284570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:26.284570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:26.284570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:26.284570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:26.284570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:26.284570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:26.284570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:26.284570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:26.284570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:26.285459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:26.285459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:26.285459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:26.285459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 435/858] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:26.290300 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf0cdf4-9403-4736-9739-e91fc89f7981 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:26.314701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:26.404993 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf0cdf4-9403-4736-9739-e91fc89f7981 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:26.443677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ce58e-e810-4b43-a5d6-40116a098e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:26.444363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ce58e-e810-4b43-a5d6-40116a098e9c None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:26.444649 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ce58e-e810-4b43-a5d6-40116a098e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:26.444921 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ce58e-e810-4b43-a5d6-40116a098e9c None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:26.445579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ce58e-e810-4b43-a5d6-40116a098e9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:26.445891 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ce58e-e810-4b43-a5d6-40116a098e9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:26.447608 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ce58e-e810-4b43-a5d6-40116a098e9c None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:26.461449 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 424/859] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:26.467289 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74140d10-cfc0-46ef-92c6-00223ad4ad66 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:26.488658 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:26.506479 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ce58e-e810-4b43-a5d6-40116a098e9c None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:26.594103 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74140d10-cfc0-46ef-92c6-00223ad4ad66 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:26.599053 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:26.623538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ce58e-e810-4b43-a5d6-40116a098e9c None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:26.644282 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc33a54-3945-4795-9f95-ecaf672de146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:26.645266 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc33a54-3945-4795-9f95-ecaf672de146 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:26.645750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc33a54-3945-4795-9f95-ecaf672de146 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:26.646231 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc33a54-3945-4795-9f95-ecaf672de146 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:26.659481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc33a54-3945-4795-9f95-ecaf672de146 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:26.661182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc33a54-3945-4795-9f95-ecaf672de146 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:26.669465 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58aad28-d21f-4f5d-85d4-ff883d24ba50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:26.670104 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58aad28-d21f-4f5d-85d4-ff883d24ba50 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:26.670641 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58aad28-d21f-4f5d-85d4-ff883d24ba50 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:26.670641 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58aad28-d21f-4f5d-85d4-ff883d24ba50 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:26.671263 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58aad28-d21f-4f5d-85d4-ff883d24ba50 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:26.672748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58aad28-d21f-4f5d-85d4-ff883d24ba50 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:26.678571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-2cc33a54-3945-4795-9f95-ecaf672de146 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 00:08:26.678571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:26.678571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:26.678571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:26.678571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:26.678571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:26.678571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:26.678571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:26.678571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:26.678571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:26.678571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:26.678571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:26.678571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:08:26.678571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:08:26.678571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:08:26.678571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:08:26.679683 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:26.679683 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:26.679683 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:08:26.679683 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:26.679683 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:08:26.679683 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:26.679683 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:26.679683 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:26.679683 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:26.679683 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:26.679683 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:26.679683 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:26.679683 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:26.679683 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:26.679683 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:26.679683 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:26.679683 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:26.679683 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:26.680620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:26.680620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:26.680620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:08:26.680620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:08:26.680620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:08:26.680620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:08:26.680620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:08:26.680620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:08:26.680620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 00:08:26.680620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:26.682044 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-a58aad28-d21f-4f5d-85d4-ff883d24ba50 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 00:08:26.682044 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:26.682044 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:08:26.682044 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:08:26.682044 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:08:26.682044 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:08:26.682044 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:08:26.682044 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:26.682044 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 00:08:26.682044 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:26.682044 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:08:26.682044 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:26.682044 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:26.682044 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:26.682044 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:26.682044 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:26.682925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:26.682925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:26.682925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:26.682925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:26.682925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:26.682925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:26.682925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:26.682925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:08:26.682925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:08:26.682925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:08:26.682925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:08:26.682925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:26.682925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:26.682925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:26.682925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:26.682925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:26.682925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:26.684092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:26.684092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:26.684092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:26.684092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:26.684092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:26.684092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:26.684092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:26.684092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:26.684092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:26.684092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:26.684092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:08:26.684092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:26.684092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 00:08:26.684092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:26.684092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 425/860] 10.222.0.67 () {56 vars in 1132 bytes} [Wed Aug 31 00:08:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:26.686717 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 436/861] 10.222.0.67 () {56 vars in 1117 bytes} [Wed Aug 31 00:08:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:26.690049 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58aad28-d21f-4f5d-85d4-ff883d24ba50 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:26.693876 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc33a54-3945-4795-9f95-ecaf672de146 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:26.799393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:26.813117 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58aad28-d21f-4f5d-85d4-ff883d24ba50 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:26.830254 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc33a54-3945-4795-9f95-ecaf672de146 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:26.861335 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001fd537-9687-457e-92f9-6f6ac8e65454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:26.862233 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001fd537-9687-457e-92f9-6f6ac8e65454 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:26.862519 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001fd537-9687-457e-92f9-6f6ac8e65454 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:26.862797 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001fd537-9687-457e-92f9-6f6ac8e65454 None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:26.863465 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001fd537-9687-457e-92f9-6f6ac8e65454 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:26.863771 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001fd537-9687-457e-92f9-6f6ac8e65454 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:26.865635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001fd537-9687-457e-92f9-6f6ac8e65454 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:26.870096 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a3e2b-1ae6-44f5-8d64-cad0eca1f900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:26.870940 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a3e2b-1ae6-44f5-8d64-cad0eca1f900 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:26.871479 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a3e2b-1ae6-44f5-8d64-cad0eca1f900 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:26.872580 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a3e2b-1ae6-44f5-8d64-cad0eca1f900 None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:26.873547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a3e2b-1ae6-44f5-8d64-cad0eca1f900 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:26.874084 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a3e2b-1ae6-44f5-8d64-cad0eca1f900 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:26.875737 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a3e2b-1ae6-44f5-8d64-cad0eca1f900 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:26.876913 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 426/862] 10.222.0.67 () {56 vars in 1126 bytes} [Wed Aug 31 00:08:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:26.877793 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001fd537-9687-457e-92f9-6f6ac8e65454 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:26.890496 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:26.902089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 437/863] 10.222.0.67 () {56 vars in 1149 bytes} [Wed Aug 31 00:08:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:26.904664 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a3e2b-1ae6-44f5-8d64-cad0eca1f900 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:27.019976 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001fd537-9687-457e-92f9-6f6ac8e65454 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:27.039889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a3e2b-1ae6-44f5-8d64-cad0eca1f900 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:27.062590 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae231af-6bd6-4f4b-8ff6-ce69f7e21ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:27.063286 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae231af-6bd6-4f4b-8ff6-ce69f7e21ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:27.063563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae231af-6bd6-4f4b-8ff6-ce69f7e21ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:27.063833 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae231af-6bd6-4f4b-8ff6-ce69f7e21ce9 None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:27.064465 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae231af-6bd6-4f4b-8ff6-ce69f7e21ce9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:27.064765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae231af-6bd6-4f4b-8ff6-ce69f7e21ce9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:27.066346 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae231af-6bd6-4f4b-8ff6-ce69f7e21ce9 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:27.071676 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 427/864] 10.222.0.67 () {56 vars in 1126 bytes} [Wed Aug 31 00:08:26 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:27.083452 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b50277-43d0-47bf-b9c8-83205dab102f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:27.084088 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b50277-43d0-47bf-b9c8-83205dab102f None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:27.084364 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b50277-43d0-47bf-b9c8-83205dab102f None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:27.084631 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b50277-43d0-47bf-b9c8-83205dab102f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:27.088743 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:27.161514 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b50277-43d0-47bf-b9c8-83205dab102f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:27.162984 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b50277-43d0-47bf-b9c8-83205dab102f None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:27.276552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 438/865] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:08:26 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:08:27.282297 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae231af-6bd6-4f4b-8ff6-ce69f7e21ce9 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:27.354083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfba623-0e45-4501-a316-ae4c94637e4b None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:27.354608 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfba623-0e45-4501-a316-ae4c94637e4b None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:27.355098 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfba623-0e45-4501-a316-ae4c94637e4b None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:27.358535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 439/866] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:27.362606 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:27.367631 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd971388-99f7-4c08-92d9-c6ce78754fcb None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:27.367631 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd971388-99f7-4c08-92d9-c6ce78754fcb None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:27.367631 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd971388-99f7-4c08-92d9-c6ce78754fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:27.389536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-bd971388-99f7-4c08-92d9-c6ce78754fcb None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:27.390795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-bd971388-99f7-4c08-92d9-c6ce78754fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:27.410379 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae231af-6bd6-4f4b-8ff6-ce69f7e21ce9 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:27.423870 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:27.459167 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e1d6f-a327-4ed9-8929-c681b8c8cad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:27.460809 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e1d6f-a327-4ed9-8929-c681b8c8cad9 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:27.461302 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e1d6f-a327-4ed9-8929-c681b8c8cad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:27.462060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e1d6f-a327-4ed9-8929-c681b8c8cad9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:27.475222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e1d6f-a327-4ed9-8929-c681b8c8cad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:27.475389 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 440/867] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:08:27.477136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e1d6f-a327-4ed9-8929-c681b8c8cad9 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:27.493466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-5d0e1d6f-a327-4ed9-8929-c681b8c8cad9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:27.493466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:27.493466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:27.493466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:27.493466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:27.493466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:27.493466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:27.493466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:27.493466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:27.493466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:27.493466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:27.493466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:27.493466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:27.493466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:27.493466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:27.493466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:27.494480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:27.494480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:27.494480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:27.494480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:27.494480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:27.494480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:27.494480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:27.494480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:27.494480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:27.494480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:27.494480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:27.494480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:27.494480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:27.494480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:27.494480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:27.494480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:27.494480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:27.495422 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:27.495422 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:27.495422 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:27.497583 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 428/868] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:27.503695 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd971388-99f7-4c08-92d9-c6ce78754fcb None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:27.610733 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd971388-99f7-4c08-92d9-c6ce78754fcb None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:27.648173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f4549-9c40-4cdf-a601-149fac1d1cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:27.649238 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f4549-9c40-4cdf-a601-149fac1d1cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:27.649727 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f4549-9c40-4cdf-a601-149fac1d1cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:27.650200 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f4549-9c40-4cdf-a601-149fac1d1cc2 None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:27.651083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f4549-9c40-4cdf-a601-149fac1d1cc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:27.651394 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f4549-9c40-4cdf-a601-149fac1d1cc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:27.652767 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f4549-9c40-4cdf-a601-149fac1d1cc2 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:27.665743 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 441/869] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:27.671773 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e1d6f-a327-4ed9-8929-c681b8c8cad9 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:27.756120 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f4549-9c40-4cdf-a601-149fac1d1cc2 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:27.786833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:27.792731 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e1d6f-a327-4ed9-8929-c681b8c8cad9 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:27.817836 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:27.830977 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39ece7-37d3-4a00-a043-7b28058ad6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:27.831668 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39ece7-37d3-4a00-a043-7b28058ad6bf None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:27.831945 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39ece7-37d3-4a00-a043-7b28058ad6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:27.832219 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39ece7-37d3-4a00-a043-7b28058ad6bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:27.839126 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39ece7-37d3-4a00-a043-7b28058ad6bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:27.840584 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39ece7-37d3-4a00-a043-7b28058ad6bf None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:27.854374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-5b39ece7-37d3-4a00-a043-7b28058ad6bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:27.854374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:27.854374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:27.854374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:27.854374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:27.854374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:27.854374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:27.854374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:27.854374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:27.854374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:27.854374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:27.854374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:27.854374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:08:27.854374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:08:27.854374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:08:27.855382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:08:27.855382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:27.855382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:27.855382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:27.855382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:27.855382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:27.855382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:27.855382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:27.855382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:27.855382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:27.855382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:27.855382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:27.855382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:27.855382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:27.855382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:27.855382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:27.855382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:27.856309 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:08:27.856309 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:08:27.856309 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:08:27.856309 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:08:27.856309 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:08:27.856309 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:08:27.856309 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:27.856309 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:27.859427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 429/870] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:27.862558 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39ece7-37d3-4a00-a043-7b28058ad6bf None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:27.891349 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f4549-9c40-4cdf-a601-149fac1d1cc2 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:27.929834 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b209f412-bc43-4627-9390-023d561a67f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:27.930552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209f412-bc43-4627-9390-023d561a67f5 None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:27.930901 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209f412-bc43-4627-9390-023d561a67f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:27.931272 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209f412-bc43-4627-9390-023d561a67f5 None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:27.932077 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b209f412-bc43-4627-9390-023d561a67f5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:27.933576 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b209f412-bc43-4627-9390-023d561a67f5 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:27.962629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 442/871] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 31 00:08:27 2022] POST /identity/v3/services => generated 251 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:08:28.009335 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39ece7-37d3-4a00-a043-7b28058ad6bf None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:28.066299 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea746ee5-51b2-44d4-8566-521538361811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:28.066917 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea746ee5-51b2-44d4-8566-521538361811 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:28.067234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea746ee5-51b2-44d4-8566-521538361811 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:28.067508 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea746ee5-51b2-44d4-8566-521538361811 None admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:28.068213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea746ee5-51b2-44d4-8566-521538361811 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:28.068581 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea746ee5-51b2-44d4-8566-521538361811 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:28.070038 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea746ee5-51b2-44d4-8566-521538361811 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:28.084960 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 430/872] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:08:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:28.091763 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b209f412-bc43-4627-9390-023d561a67f5 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:28.190238 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:28.208655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b209f412-bc43-4627-9390-023d561a67f5 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:28.248961 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbceef4-742b-42db-83b2-e99adc96ed6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:28.249780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbceef4-742b-42db-83b2-e99adc96ed6e None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:28.250069 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbceef4-742b-42db-83b2-e99adc96ed6e None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:28.250345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbceef4-742b-42db-83b2-e99adc96ed6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:28.251496 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbceef4-742b-42db-83b2-e99adc96ed6e None admin] RBAC: Adding query filter params (role.id=4680aa64da80470b87075090fca2354b, scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb, user.id=a8fb7a9acefd4b418c992661037c2ed9) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:28.251809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbceef4-742b-42db-83b2-e99adc96ed6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:28.253334 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbceef4-742b-42db-83b2-e99adc96ed6e None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:28.260470 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 443/873] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:08:28 2022] GET /identity/v3/role_assignments?user.id=a8fb7a9acefd4b418c992661037c2ed9&scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb&role.id=4680aa64da80470b87075090fca2354b => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:28.274654 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8db687-df29-47a6-a33a-119c5502b2c5 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:28.274995 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8db687-df29-47a6-a33a-119c5502b2c5 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:28.275250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8db687-df29-47a6-a33a-119c5502b2c5 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:28.278175 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 431/874] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:28.284413 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6d4e5-e8f0-444e-9859-1080c075e600 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:28.284727 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6d4e5-e8f0-444e-9859-1080c075e600 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:28.284975 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6d4e5-e8f0-444e-9859-1080c075e600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:28.307204 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-b9e6d4e5-e8f0-444e-9859-1080c075e600 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:28.308418 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b9e6d4e5-e8f0-444e-9859-1080c075e600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:28.396334 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 444/875] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:28 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:28.671054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8db687-df29-47a6-a33a-119c5502b2c5 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:28.720209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:28.789286 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8db687-df29-47a6-a33a-119c5502b2c5 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:28.826021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f674f-1f0f-4f0b-8a1e-4c8dd4f6e83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:28.826644 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f674f-1f0f-4f0b-8a1e-4c8dd4f6e83f None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:28.826920 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f674f-1f0f-4f0b-8a1e-4c8dd4f6e83f None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:28.827221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f674f-1f0f-4f0b-8a1e-4c8dd4f6e83f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:28.827806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f674f-1f0f-4f0b-8a1e-4c8dd4f6e83f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:28.829162 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f674f-1f0f-4f0b-8a1e-4c8dd4f6e83f None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:28.838337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-c95f674f-1f0f-4f0b-8a1e-4c8dd4f6e83f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:08:28.838337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:28.838337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:08:28.838337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:08:28.838337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:08:28.838337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:08:28.838337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:08:28.838337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:28.838337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:08:28.838337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:28.838337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:08:28.838337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:28.838337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:28.838337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:28.838337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:28.838337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:28.838337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:28.839350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:28.839350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:28.839350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:28.839350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:28.839350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:28.839350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:28.839350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:08:28.839350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:08:28.839350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:08:28.839350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:08:28.839350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:28.839350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:28.839350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:28.839350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:28.839350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:28.839350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:28.840181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:28.840181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:28.840181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:28.840181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:28.840181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:28.840181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:28.840181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:28.840181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:28.840181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:28.840181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:28.840181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:08:28.840181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:28.840181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:08:28.840181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:28.843630 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 432/876] 10.222.0.67 () {56 vars in 1126 bytes} [Wed Aug 31 00:08:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:28.845909 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6d4e5-e8f0-444e-9859-1080c075e600 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:28.936723 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:28.957974 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6d4e5-e8f0-444e-9859-1080c075e600 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:28.999105 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1efd6-0042-4830-ba0a-2d7abd764c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:28.999105 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1efd6-0042-4830-ba0a-2d7abd764c24 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:28.999105 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1efd6-0042-4830-ba0a-2d7abd764c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:28.999105 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1efd6-0042-4830-ba0a-2d7abd764c24 None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:28.999580 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a1efd6-0042-4830-ba0a-2d7abd764c24 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:28.999580 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a1efd6-0042-4830-ba0a-2d7abd764c24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:28.999580 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a1efd6-0042-4830-ba0a-2d7abd764c24 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:29.009257 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 445/877] 10.222.0.67 () {56 vars in 1122 bytes} [Wed Aug 31 00:08:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:29.010751 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f674f-1f0f-4f0b-8a1e-4c8dd4f6e83f None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:29.123940 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f674f-1f0f-4f0b-8a1e-4c8dd4f6e83f None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:29.125542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:29.135798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c1d65-6089-4670-a638-bff8531df647 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:29.136088 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c1d65-6089-4670-a638-bff8531df647 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:29.136365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c1d65-6089-4670-a638-bff8531df647 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:29.138127 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 446/878] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:08:29.144013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a110a-4780-43c2-bffe-16f3c170b482 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:29.144573 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a110a-4780-43c2-bffe-16f3c170b482 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:29.145047 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a110a-4780-43c2-bffe-16f3c170b482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:29.164316 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-b95a110a-4780-43c2-bffe-16f3c170b482 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:29.166107 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b95a110a-4780-43c2-bffe-16f3c170b482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:29.166968 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc410b39-04de-4c8a-b4ab-001e3be8ed0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:29.167611 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc410b39-04de-4c8a-b4ab-001e3be8ed0b None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:29.167892 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc410b39-04de-4c8a-b4ab-001e3be8ed0b None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:29.168166 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc410b39-04de-4c8a-b4ab-001e3be8ed0b None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:29.168797 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc410b39-04de-4c8a-b4ab-001e3be8ed0b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:29.169805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc410b39-04de-4c8a-b4ab-001e3be8ed0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:29.172236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc410b39-04de-4c8a-b4ab-001e3be8ed0b None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:29.179110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 433/879] 10.222.0.67 () {56 vars in 1122 bytes} [Wed Aug 31 00:08:29 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:29.253337 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 447/880] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:29 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:08:29.531735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc410b39-04de-4c8a-b4ab-001e3be8ed0b None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:29.643250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc410b39-04de-4c8a-b4ab-001e3be8ed0b None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:29.681519 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823a180-9273-4ca4-a317-5e794de1df2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:29.682128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823a180-9273-4ca4-a317-5e794de1df2e None admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:29.682404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823a180-9273-4ca4-a317-5e794de1df2e None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:29.682681 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823a180-9273-4ca4-a317-5e794de1df2e None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:29.683378 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1823a180-9273-4ca4-a317-5e794de1df2e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:29.684868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1823a180-9273-4ca4-a317-5e794de1df2e None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:29.703396 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 434/881] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 31 00:08:29 2022] POST /identity/v3/services => generated 269 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:08:30.095017 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bd402-272d-4732-8137-21d31fa66bcf None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:30.095544 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bd402-272d-4732-8137-21d31fa66bcf None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:30.095654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bd402-272d-4732-8137-21d31fa66bcf None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:30.097184 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 448/882] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:30.102363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605aaef-a54c-4149-80b3-f96504883311 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:30.102775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605aaef-a54c-4149-80b3-f96504883311 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:30.102918 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605aaef-a54c-4149-80b3-f96504883311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:30.122560 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-8605aaef-a54c-4149-80b3-f96504883311 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:30.124016 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-8605aaef-a54c-4149-80b3-f96504883311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:30.147166 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:30.209277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 435/883] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:30 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:30.431797 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f86d0-0d02-48a8-aecc-5d801c68a7d6 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:30.432168 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f86d0-0d02-48a8-aecc-5d801c68a7d6 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:30.432373 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f86d0-0d02-48a8-aecc-5d801c68a7d6 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:30.433901 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 449/884] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:30.439183 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695b388-0a99-45fe-bd1c-303bb90a31f5 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:30.439469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695b388-0a99-45fe-bd1c-303bb90a31f5 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:30.439737 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695b388-0a99-45fe-bd1c-303bb90a31f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:30.458594 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-6695b388-0a99-45fe-bd1c-303bb90a31f5 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:30.459798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-6695b388-0a99-45fe-bd1c-303bb90a31f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:30.485396 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f86d0-0d02-48a8-aecc-5d801c68a7d6 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:30.534271 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 436/885] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:30 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:30.580568 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:30.608349 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f86d0-0d02-48a8-aecc-5d801c68a7d6 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:30.644938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebda146-eaae-4174-861e-0776af6b52e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:30.645573 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebda146-eaae-4174-861e-0776af6b52e2 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:30.645854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebda146-eaae-4174-861e-0776af6b52e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:30.646126 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebda146-eaae-4174-861e-0776af6b52e2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:30.646706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebda146-eaae-4174-861e-0776af6b52e2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:30.648115 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebda146-eaae-4174-861e-0776af6b52e2 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:30.657173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-aebda146-eaae-4174-861e-0776af6b52e2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 00:08:30.657173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:30.657173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:08:30.657173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:08:30.657173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:08:30.657173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:08:30.657173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:08:30.657173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:30.657173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 00:08:30.657173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:30.657173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:08:30.657173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:30.657173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:30.657173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:30.657173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:30.657173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:30.657173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:30.658195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:30.658195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:30.658195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:30.658195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:30.658195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:30.658195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:30.658195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:08:30.658195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:08:30.658195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:08:30.658195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:08:30.658195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:30.658195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:30.658195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:30.658195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:30.658195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:30.658195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:30.660063 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:30.660063 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:30.660063 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:30.660063 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:30.660063 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:30.660063 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:30.660063 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:30.660063 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:30.660063 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:30.660063 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:30.660063 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:08:30.660063 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:30.660063 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 00:08:30.660063 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:30.660063 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 450/886] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 31 00:08:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:30.664060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695b388-0a99-45fe-bd1c-303bb90a31f5 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:30.754459 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:30.774464 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695b388-0a99-45fe-bd1c-303bb90a31f5 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:30.807334 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebda146-eaae-4174-861e-0776af6b52e2 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:30.808723 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506f259-6a29-437e-81ad-63bd29830050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:30.809339 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506f259-6a29-437e-81ad-63bd29830050 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:30.810157 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506f259-6a29-437e-81ad-63bd29830050 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:30.810432 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506f259-6a29-437e-81ad-63bd29830050 None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:30.811282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506f259-6a29-437e-81ad-63bd29830050 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:30.812453 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506f259-6a29-437e-81ad-63bd29830050 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:30.813903 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506f259-6a29-437e-81ad-63bd29830050 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:30.819477 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 437/887] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 31 00:08:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:30.824860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506f259-6a29-437e-81ad-63bd29830050 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:30.942252 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebda146-eaae-4174-861e-0776af6b52e2 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:30.947155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506f259-6a29-437e-81ad-63bd29830050 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:30.985886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c2a011-40a5-4058-b400-b8bd1c0a99d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:30.986741 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2a011-40a5-4058-b400-b8bd1c0a99d7 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:30.987197 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2a011-40a5-4058-b400-b8bd1c0a99d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:30.987627 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2a011-40a5-4058-b400-b8bd1c0a99d7 None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:30.988623 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c2a011-40a5-4058-b400-b8bd1c0a99d7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:30.989216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c2a011-40a5-4058-b400-b8bd1c0a99d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:30.990767 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c2a011-40a5-4058-b400-b8bd1c0a99d7 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:30.999123 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 438/888] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 31 00:08:30 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:31.003678 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c2a011-40a5-4058-b400-b8bd1c0a99d7 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:31.005593 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de1eb3-586f-4575-8660-146fedbdee5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:31.006180 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de1eb3-586f-4575-8660-146fedbdee5f None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:31.006462 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de1eb3-586f-4575-8660-146fedbdee5f None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:31.006737 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de1eb3-586f-4575-8660-146fedbdee5f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:31.017228 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:31.019352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:31.024248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de1eb3-586f-4575-8660-146fedbdee5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:08:31.030065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-15de1eb3-586f-4575-8660-146fedbdee5f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 00:08:31.030065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:31.030065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:31.030065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:31.030065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:31.030065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:31.030065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:31.030065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:31.030065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:31.030065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:31.030065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:31.030065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:31.030065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:08:31.030065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:08:31.030065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:08:31.030065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:08:31.030990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:08:31.030990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:08:31.030990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:08:31.030990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:08:31.030990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:31.030990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:31.030990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:31.030990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:31.030990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:31.030990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:31.030990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:31.030990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:31.030990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:31.030990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:31.030990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:31.030990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:31.031839 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:31.031839 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:31.031839 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:31.031839 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:31.031839 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:08:31.031839 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:08:31.031839 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:08:31.031839 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:08:31.031839 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:08:31.031839 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:08:31.031839 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 00:08:31.031839 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:31.034448 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 451/889] 10.222.0.67 () {56 vars in 1120 bytes} [Wed Aug 31 00:08:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:31.040000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de1eb3-586f-4575-8660-146fedbdee5f None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:31.144552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c2a011-40a5-4058-b400-b8bd1c0a99d7 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:31.169536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de1eb3-586f-4575-8660-146fedbdee5f None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:31.194298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f95ed8-ed9c-4ad2-865f-13abe40e740a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:31.194919 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f95ed8-ed9c-4ad2-865f-13abe40e740a None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:31.195231 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f95ed8-ed9c-4ad2-865f-13abe40e740a None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:31.195507 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f95ed8-ed9c-4ad2-865f-13abe40e740a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:31.196756 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f95ed8-ed9c-4ad2-865f-13abe40e740a None admin] RBAC: Adding query filter params (interface=public, service_id=27a3844844494e1a82422dc6dd1e5aba, region_id=RegionOne) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:31.196869 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f95ed8-ed9c-4ad2-865f-13abe40e740a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:31.198933 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f95ed8-ed9c-4ad2-865f-13abe40e740a None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:31.207102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 439/890] 10.222.0.67 () {56 vars in 1251 bytes} [Wed Aug 31 00:08:31 2022] GET /identity/v3/endpoints?service_id=27a3844844494e1a82422dc6dd1e5aba&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:31.212882 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f95ed8-ed9c-4ad2-865f-13abe40e740a None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:31.213795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d444c-081a-4ae2-88eb-73a6f194e4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:31.214638 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d444c-081a-4ae2-88eb-73a6f194e4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:31.214980 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d444c-081a-4ae2-88eb-73a6f194e4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:31.215360 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d444c-081a-4ae2-88eb-73a6f194e4b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:31.216777 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d444c-081a-4ae2-88eb-73a6f194e4b2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:31.217149 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d444c-081a-4ae2-88eb-73a6f194e4b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:31.218676 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d444c-081a-4ae2-88eb-73a6f194e4b2 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:31.227952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 452/891] 10.222.0.67 () {56 vars in 1115 bytes} [Wed Aug 31 00:08:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:31.234248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d444c-081a-4ae2-88eb-73a6f194e4b2 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:31.299192 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:31.300377 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:31.329755 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f95ed8-ed9c-4ad2-865f-13abe40e740a None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:31.344508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d444c-081a-4ae2-88eb-73a6f194e4b2 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:31.365064 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc794a3-7442-42a6-8ad4-5d08cd0f3a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:31.365678 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc794a3-7442-42a6-8ad4-5d08cd0f3a30 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:31.365958 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc794a3-7442-42a6-8ad4-5d08cd0f3a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:31.366227 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc794a3-7442-42a6-8ad4-5d08cd0f3a30 None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:31.366830 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc794a3-7442-42a6-8ad4-5d08cd0f3a30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:31.368201 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc794a3-7442-42a6-8ad4-5d08cd0f3a30 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:31.384292 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 440/892] 10.222.0.67 () {56 vars in 1087 bytes} [Wed Aug 31 00:08:31 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:08:31.384292 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620110a-bda4-48ee-9162-f832a740e044 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:31.384292 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620110a-bda4-48ee-9162-f832a740e044 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:31.384292 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620110a-bda4-48ee-9162-f832a740e044 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:31.384292 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 441/893] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:31.386989 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e975790-46c7-411a-a360-34eaff60a519 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:31.387305 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e975790-46c7-411a-a360-34eaff60a519 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:31.387632 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e975790-46c7-411a-a360-34eaff60a519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:31.396821 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b918e08-513f-4688-bc9f-70ff74fe3304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:31.397869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b918e08-513f-4688-bc9f-70ff74fe3304 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:31.398369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b918e08-513f-4688-bc9f-70ff74fe3304 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:31.398862 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b918e08-513f-4688-bc9f-70ff74fe3304 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:31.408151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-0e975790-46c7-411a-a360-34eaff60a519 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:31.409363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-0e975790-46c7-411a-a360-34eaff60a519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:31.412834 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b918e08-513f-4688-bc9f-70ff74fe3304 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:31.414396 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b918e08-513f-4688-bc9f-70ff74fe3304 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:31.433058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-6b918e08-513f-4688-bc9f-70ff74fe3304 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:31.433058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:31.433058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:31.433058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:31.433058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:31.433058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:31.433058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:31.433058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:31.433058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:31.433058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:31.433058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:31.433058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:31.433058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:31.433058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:31.433058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:31.433058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:31.434140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:31.434140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:31.434140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:31.434140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:31.434140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:31.434140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:31.434140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:31.434140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:31.434140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:31.434140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:31.434140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:31.434140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:31.434140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:31.434140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:31.434140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:31.434140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:31.434140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:31.436211 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:31.436211 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:31.436211 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:31.439220 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 453/894] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:31.441954 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b918e08-513f-4688-bc9f-70ff74fe3304 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:31.523087 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 442/895] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:31.589381 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b918e08-513f-4688-bc9f-70ff74fe3304 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:31.605033 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:31.629159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a1e29-e4c6-40d6-b3c5-c145a3c5f3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:31.629774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a1e29-e4c6-40d6-b3c5-c145a3c5f3c6 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:31.630104 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a1e29-e4c6-40d6-b3c5-c145a3c5f3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:31.630380 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a1e29-e4c6-40d6-b3c5-c145a3c5f3c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:31.631023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a1e29-e4c6-40d6-b3c5-c145a3c5f3c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:31.631351 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a1e29-e4c6-40d6-b3c5-c145a3c5f3c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:31.632721 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a1e29-e4c6-40d6-b3c5-c145a3c5f3c6 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:31.645837 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 454/896] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:31.651062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e975790-46c7-411a-a360-34eaff60a519 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:31.722640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:31.752221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbe625-d69c-44b1-9790-4281b243fa7b None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:31.752568 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbe625-d69c-44b1-9790-4281b243fa7b None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:31.752781 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbe625-d69c-44b1-9790-4281b243fa7b None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:31.754477 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 455/897] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:31.757708 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e975790-46c7-411a-a360-34eaff60a519 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:31.759522 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410163ca-cafb-42f9-a32f-49daefc01c95 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:31.759811 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410163ca-cafb-42f9-a32f-49daefc01c95 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:31.760183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410163ca-cafb-42f9-a32f-49daefc01c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:31.780399 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-410163ca-cafb-42f9-a32f-49daefc01c95 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:31.781604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-410163ca-cafb-42f9-a32f-49daefc01c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:31.793738 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50811514-f93d-4455-a47b-58f3f515be61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:31.794913 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50811514-f93d-4455-a47b-58f3f515be61 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:31.795253 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50811514-f93d-4455-a47b-58f3f515be61 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:31.795477 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50811514-f93d-4455-a47b-58f3f515be61 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:31.803126 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50811514-f93d-4455-a47b-58f3f515be61 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:31.804582 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50811514-f93d-4455-a47b-58f3f515be61 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:31.817093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-50811514-f93d-4455-a47b-58f3f515be61 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 00:08:31.817093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:31.817093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:31.817093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:31.817093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:31.817093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:31.817093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:31.817093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:31.817093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:31.817093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:31.817093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:31.817093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:31.817093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:08:31.817093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:08:31.817093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:08:31.817093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:08:31.818110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:31.818110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:31.818110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:08:31.818110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:31.818110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:08:31.818110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:31.818110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:31.818110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:31.818110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:31.818110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:31.818110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:31.818110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:31.818110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:31.818110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:31.818110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:31.818110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:31.818110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:31.818110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:31.819105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:31.819105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:31.819105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:08:31.819105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:08:31.819105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:08:31.819105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:08:31.819105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:08:31.819105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:08:31.819105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 00:08:31.819105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:31.819105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 443/898] 10.222.0.67 () {56 vars in 1117 bytes} [Wed Aug 31 00:08:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:31.819730 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50811514-f93d-4455-a47b-58f3f515be61 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:31.865994 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 456/899] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:08:31.866858 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410163ca-cafb-42f9-a32f-49daefc01c95 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:31.944187 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50811514-f93d-4455-a47b-58f3f515be61 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:31.979680 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:31.990202 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5218ad-4504-4bde-b30d-9f2aff2cdd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:31.990817 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5218ad-4504-4bde-b30d-9f2aff2cdd14 None admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:31.991119 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5218ad-4504-4bde-b30d-9f2aff2cdd14 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:31.991389 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5218ad-4504-4bde-b30d-9f2aff2cdd14 None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:31.993181 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410163ca-cafb-42f9-a32f-49daefc01c95 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:31.993824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5218ad-4504-4bde-b30d-9f2aff2cdd14 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:31.995265 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5218ad-4504-4bde-b30d-9f2aff2cdd14 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:31.996038 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:32.023655 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 444/900] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 31 00:08:31 2022] POST /identity/v3/services => generated 241 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:08:32.041820 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e356d-2539-4f1f-9d13-27bfeb2bc391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:32.042995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e356d-2539-4f1f-9d13-27bfeb2bc391 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:32.043529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e356d-2539-4f1f-9d13-27bfeb2bc391 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:32.044124 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e356d-2539-4f1f-9d13-27bfeb2bc391 None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:32.045536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e356d-2539-4f1f-9d13-27bfeb2bc391 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:32.046097 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e356d-2539-4f1f-9d13-27bfeb2bc391 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:32.047949 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e356d-2539-4f1f-9d13-27bfeb2bc391 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:32.070709 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 457/901] 10.222.0.67 () {56 vars in 1149 bytes} [Wed Aug 31 00:08:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:32.076960 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5218ad-4504-4bde-b30d-9f2aff2cdd14 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:32.150185 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e356d-2539-4f1f-9d13-27bfeb2bc391 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:32.206434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5218ad-4504-4bde-b30d-9f2aff2cdd14 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:32.245219 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69add1ab-5cc9-43d6-95ca-b83ec38a6c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:32.246241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69add1ab-5cc9-43d6-95ca-b83ec38a6c20 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:32.246736 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69add1ab-5cc9-43d6-95ca-b83ec38a6c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:32.248255 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69add1ab-5cc9-43d6-95ca-b83ec38a6c20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:32.281902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e356d-2539-4f1f-9d13-27bfeb2bc391 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:32.316940 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:32.331103 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed62c3-f0bb-4432-8e1a-fb12adc0e6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:32.333029 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed62c3-f0bb-4432-8e1a-fb12adc0e6b6 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:32.333313 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed62c3-f0bb-4432-8e1a-fb12adc0e6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:32.333585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed62c3-f0bb-4432-8e1a-fb12adc0e6b6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:32.334176 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed62c3-f0bb-4432-8e1a-fb12adc0e6b6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:32.335595 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed62c3-f0bb-4432-8e1a-fb12adc0e6b6 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:32.336240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69add1ab-5cc9-43d6-95ca-b83ec38a6c20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:32.338628 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69add1ab-5cc9-43d6-95ca-b83ec38a6c20 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:32.345872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-3eed62c3-f0bb-4432-8e1a-fb12adc0e6b6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 00:08:32.345872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:32.345872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:08:32.345872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:08:32.345872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:08:32.345872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:08:32.345872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:08:32.345872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:32.345872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 00:08:32.345872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:32.345872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:08:32.345872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:32.345872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:32.345872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:32.345872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:32.345872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:32.346925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:32.346925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:32.346925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:32.346925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:32.346925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:32.346925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:32.346925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:32.346925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:08:32.346925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:08:32.346925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:08:32.346925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:08:32.346925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:32.346925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:32.346925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:32.346925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:32.346925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:32.346925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:32.347899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:32.347899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:32.347899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:32.347899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:32.347899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:32.347899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:32.347899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:32.347899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:32.347899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:32.347899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:32.347899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:08:32.347899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:32.347899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 00:08:32.347899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:32.347899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 458/902] 10.222.0.67 () {56 vars in 1132 bytes} [Wed Aug 31 00:08:32 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:32.354676 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed62c3-f0bb-4432-8e1a-fb12adc0e6b6 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:32.383061 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:32.469058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 445/903] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:08:32 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 00:08:32.474162 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69add1ab-5cc9-43d6-95ca-b83ec38a6c20 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:32.489537 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed62c3-f0bb-4432-8e1a-fb12adc0e6b6 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:32.535186 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af94e18-d094-4ce2-90a4-2e919d853b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:32.536407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af94e18-d094-4ce2-90a4-2e919d853b3b None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:32.537138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af94e18-d094-4ce2-90a4-2e919d853b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:32.538017 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af94e18-d094-4ce2-90a4-2e919d853b3b None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:32.539176 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af94e18-d094-4ce2-90a4-2e919d853b3b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:32.539692 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af94e18-d094-4ce2-90a4-2e919d853b3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:32.542176 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af94e18-d094-4ce2-90a4-2e919d853b3b None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:32.555016 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 459/904] 10.222.0.67 () {56 vars in 1126 bytes} [Wed Aug 31 00:08:32 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:32.556890 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af94e18-d094-4ce2-90a4-2e919d853b3b None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:32.580141 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:32.597545 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69add1ab-5cc9-43d6-95ca-b83ec38a6c20 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:32.642599 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6374b0-6db2-42fb-bdd2-8b52d334b7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:32.643617 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6374b0-6db2-42fb-bdd2-8b52d334b7a4 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:32.644077 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6374b0-6db2-42fb-bdd2-8b52d334b7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:32.644540 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6374b0-6db2-42fb-bdd2-8b52d334b7a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:32.656293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6374b0-6db2-42fb-bdd2-8b52d334b7a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:32.658007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6374b0-6db2-42fb-bdd2-8b52d334b7a4 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:32.671278 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:32.676060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-8c6374b0-6db2-42fb-bdd2-8b52d334b7a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:32.676060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:32.676060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:32.676060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:32.676060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:32.676060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:32.676060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:32.676060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:32.676060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:32.676060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:32.676060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:32.676060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:32.676060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:32.676060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:32.676060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:32.676060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:32.680148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:32.680148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:32.680148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:32.680148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:32.680148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:32.680148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:32.680148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:32.680148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:32.680148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:32.680148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:32.680148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:32.680148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:32.680148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:32.680148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:32.680148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:32.680148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:32.680148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:32.681027 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:32.681027 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:32.681027 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:32.682612 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 446/905] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:32.683591 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6374b0-6db2-42fb-bdd2-8b52d334b7a4 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:32.690281 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af94e18-d094-4ce2-90a4-2e919d853b3b None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:32.740518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f6dff-68fb-4df0-8f07-512192b8c61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:32.741046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f6dff-68fb-4df0-8f07-512192b8c61e None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:32.741697 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f6dff-68fb-4df0-8f07-512192b8c61e None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:32.741975 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f6dff-68fb-4df0-8f07-512192b8c61e None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:32.742602 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f6dff-68fb-4df0-8f07-512192b8c61e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:32.742904 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f6dff-68fb-4df0-8f07-512192b8c61e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:32.745029 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f6dff-68fb-4df0-8f07-512192b8c61e None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:32.755126 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 460/906] 10.222.0.67 () {56 vars in 1126 bytes} [Wed Aug 31 00:08:32 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:32.758519 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f6dff-68fb-4df0-8f07-512192b8c61e None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:32.806016 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6374b0-6db2-42fb-bdd2-8b52d334b7a4 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:32.852705 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4fd88-ef02-419a-a893-d31c304cc3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:32.853693 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4fd88-ef02-419a-a893-d31c304cc3d4 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:32.854221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4fd88-ef02-419a-a893-d31c304cc3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:32.854716 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4fd88-ef02-419a-a893-d31c304cc3d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:32.856042 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4fd88-ef02-419a-a893-d31c304cc3d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:32.857686 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4fd88-ef02-419a-a893-d31c304cc3d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:32.859714 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4fd88-ef02-419a-a893-d31c304cc3d4 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:32.872320 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:32.881261 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 447/907] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:32.882272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4fd88-ef02-419a-a893-d31c304cc3d4 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:32.885982 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f6dff-68fb-4df0-8f07-512192b8c61e None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:32.932682 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3b158-3cc7-4cb3-900d-abe3d4e530f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:32.933982 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3b158-3cc7-4cb3-900d-abe3d4e530f3 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:32.935279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3b158-3cc7-4cb3-900d-abe3d4e530f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:32.935766 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3b158-3cc7-4cb3-900d-abe3d4e530f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:32.936843 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a3b158-3cc7-4cb3-900d-abe3d4e530f3 None admin] RBAC: Adding query filter params (interface=public, service_id=228c3666c6d2470e910fb1b43bea2d4b, region_id=RegionOne) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:32.937634 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a3b158-3cc7-4cb3-900d-abe3d4e530f3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:32.939238 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a3b158-3cc7-4cb3-900d-abe3d4e530f3 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:32.951648 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 461/908] 10.222.0.67 () {56 vars in 1251 bytes} [Wed Aug 31 00:08:32 2022] GET /identity/v3/endpoints?service_id=228c3666c6d2470e910fb1b43bea2d4b&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:32.952824 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3b158-3cc7-4cb3-900d-abe3d4e530f3 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:32.969015 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:32.994594 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4fd88-ef02-419a-a893-d31c304cc3d4 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:33.033331 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731351b8-7623-48e1-9e75-9a33cdbe0dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:33.034059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731351b8-7623-48e1-9e75-9a33cdbe0dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:33.034411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731351b8-7623-48e1-9e75-9a33cdbe0dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:33.034755 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731351b8-7623-48e1-9e75-9a33cdbe0dc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:33.042504 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731351b8-7623-48e1-9e75-9a33cdbe0dc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:33.044171 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731351b8-7623-48e1-9e75-9a33cdbe0dc1 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:33.056223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-731351b8-7623-48e1-9e75-9a33cdbe0dc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:33.056223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:33.056223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:33.056223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:33.056223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:33.056223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:33.056223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:33.056223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:33.056223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:33.056223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:33.056223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:33.056223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:33.056223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:08:33.056223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:08:33.056223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:08:33.057160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:08:33.057160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:33.057160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:33.057160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:33.057160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:33.057160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:33.057160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:33.057160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:33.057160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:33.057160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:33.057160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:33.057160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:33.057160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:33.057160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:33.057160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:33.057160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:33.057160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:33.058070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:08:33.058070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:08:33.058070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:08:33.058070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:08:33.058070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:08:33.058070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:08:33.058070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:33.058070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:33.062768 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 448/909] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:33.063942 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731351b8-7623-48e1-9e75-9a33cdbe0dc1 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:33.069875 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3b158-3cc7-4cb3-900d-abe3d4e530f3 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:33.106759 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0ce78-130e-40e3-82b4-6834b27fa8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:33.107966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0ce78-130e-40e3-82b4-6834b27fa8bb None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:33.108465 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0ce78-130e-40e3-82b4-6834b27fa8bb None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:33.108952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0ce78-130e-40e3-82b4-6834b27fa8bb None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:33.109774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d0ce78-130e-40e3-82b4-6834b27fa8bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:33.111368 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d0ce78-130e-40e3-82b4-6834b27fa8bb None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:33.117880 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 462/910] 10.222.0.67 () {56 vars in 1087 bytes} [Wed Aug 31 00:08:32 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:08:33.129444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:33.180453 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731351b8-7623-48e1-9e75-9a33cdbe0dc1 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:33.219701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8277b1d-b40f-4bd2-ad34-c50b19f9a12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:33.220372 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8277b1d-b40f-4bd2-ad34-c50b19f9a12a None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:33.220659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8277b1d-b40f-4bd2-ad34-c50b19f9a12a None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:33.220935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8277b1d-b40f-4bd2-ad34-c50b19f9a12a None admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:33.221705 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8277b1d-b40f-4bd2-ad34-c50b19f9a12a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:33.222018 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8277b1d-b40f-4bd2-ad34-c50b19f9a12a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:33.223544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8277b1d-b40f-4bd2-ad34-c50b19f9a12a None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:33.243199 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 449/911] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:08:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:33.248313 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0ce78-130e-40e3-82b4-6834b27fa8bb None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:33.338044 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:33.365247 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0ce78-130e-40e3-82b4-6834b27fa8bb None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:33.402599 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242196e1-7c74-4e30-94db-6004d173e993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:33.403701 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242196e1-7c74-4e30-94db-6004d173e993 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:33.404237 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242196e1-7c74-4e30-94db-6004d173e993 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:33.404789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242196e1-7c74-4e30-94db-6004d173e993 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:33.405723 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242196e1-7c74-4e30-94db-6004d173e993 None admin] RBAC: Adding query filter params (role.id=39f496940d6844c8a43185fbd5fc1d21, scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb, user.id=a8fb7a9acefd4b418c992661037c2ed9) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:33.406234 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242196e1-7c74-4e30-94db-6004d173e993 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:33.407900 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242196e1-7c74-4e30-94db-6004d173e993 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:33.414183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 463/912] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:08:33 2022] GET /identity/v3/role_assignments?user.id=a8fb7a9acefd4b418c992661037c2ed9&scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb&role.id=39f496940d6844c8a43185fbd5fc1d21 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:33.527404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654192fa-aa44-4982-92a2-23c88987ab86 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:33.527598 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654192fa-aa44-4982-92a2-23c88987ab86 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:33.527872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654192fa-aa44-4982-92a2-23c88987ab86 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:33.530146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 450/913] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:33.535945 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c0957-5b7d-422c-bcf9-4a0ae2cbad73 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:33.536547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c0957-5b7d-422c-bcf9-4a0ae2cbad73 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:33.537026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c0957-5b7d-422c-bcf9-4a0ae2cbad73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:33.557899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-627c0957-5b7d-422c-bcf9-4a0ae2cbad73 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:33.560038 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-627c0957-5b7d-422c-bcf9-4a0ae2cbad73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:33.631787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 464/914] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:33.907837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654192fa-aa44-4982-92a2-23c88987ab86 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:34.019871 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654192fa-aa44-4982-92a2-23c88987ab86 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:34.042080 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:34.057332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2210b35-36b6-46e4-ad66-d5090fc06e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:34.058402 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2210b35-36b6-46e4-ad66-d5090fc06e77 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:34.058899 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2210b35-36b6-46e4-ad66-d5090fc06e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:34.059430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2210b35-36b6-46e4-ad66-d5090fc06e77 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:34.060964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2210b35-36b6-46e4-ad66-d5090fc06e77 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:34.062533 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2210b35-36b6-46e4-ad66-d5090fc06e77 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:34.071954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-f2210b35-36b6-46e4-ad66-d5090fc06e77 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 00:08:34.071954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:34.071954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:08:34.071954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:08:34.071954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:08:34.071954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:08:34.071954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:08:34.071954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:34.071954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 00:08:34.071954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:34.071954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:08:34.071954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:34.071954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:34.071954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:34.071954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:34.071954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:34.071954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:34.072943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:34.072943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:34.072943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:34.072943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:34.072943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:34.072943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:34.072943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:08:34.072943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:08:34.072943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:08:34.072943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:08:34.072943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:34.072943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:34.072943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:34.072943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:34.072943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:34.072943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:34.073777 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:34.073777 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:34.073777 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:34.073777 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:34.073777 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:34.073777 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:34.073777 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:34.073777 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:34.073777 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:34.073777 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:34.073777 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:08:34.073777 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:34.073777 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 00:08:34.073777 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:34.077010 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 451/915] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 31 00:08:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:34.084709 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c0957-5b7d-422c-bcf9-4a0ae2cbad73 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:34.092995 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555f36a-a55e-4110-ad74-7c3ed7e882f2 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:34.093188 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555f36a-a55e-4110-ad74-7c3ed7e882f2 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:34.093289 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555f36a-a55e-4110-ad74-7c3ed7e882f2 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:34.094869 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 452/916] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:34.099725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ac437-0ef1-43bb-8cd3-ca8bf37d18a1 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:34.100004 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ac437-0ef1-43bb-8cd3-ca8bf37d18a1 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:34.100270 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ac437-0ef1-43bb-8cd3-ca8bf37d18a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:34.120132 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-491ac437-0ef1-43bb-8cd3-ca8bf37d18a1 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:34.121336 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-491ac437-0ef1-43bb-8cd3-ca8bf37d18a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:34.171950 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:34.206567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 453/917] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:34.217332 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c0957-5b7d-422c-bcf9-4a0ae2cbad73 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:34.254517 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77d7a3-5b50-48ee-bb01-a442b9e138cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:34.255159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77d7a3-5b50-48ee-bb01-a442b9e138cb None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:34.255445 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77d7a3-5b50-48ee-bb01-a442b9e138cb None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:34.255719 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77d7a3-5b50-48ee-bb01-a442b9e138cb None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:34.256348 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77d7a3-5b50-48ee-bb01-a442b9e138cb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:34.256651 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77d7a3-5b50-48ee-bb01-a442b9e138cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:34.258021 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77d7a3-5b50-48ee-bb01-a442b9e138cb None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:34.263918 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 465/918] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 31 00:08:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:34.269730 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491ac437-0ef1-43bb-8cd3-ca8bf37d18a1 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:34.375384 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491ac437-0ef1-43bb-8cd3-ca8bf37d18a1 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:34.392303 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:34.413456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd298a4-0bdb-4178-bc03-4bbe60ba8fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:34.414065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd298a4-0bdb-4178-bc03-4bbe60ba8fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:34.414338 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd298a4-0bdb-4178-bc03-4bbe60ba8fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:34.414599 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd298a4-0bdb-4178-bc03-4bbe60ba8fa3 None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:34.415256 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd298a4-0bdb-4178-bc03-4bbe60ba8fa3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:34.415552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd298a4-0bdb-4178-bc03-4bbe60ba8fa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:34.416898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd298a4-0bdb-4178-bc03-4bbe60ba8fa3 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:34.422766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 454/919] 10.222.0.67 () {56 vars in 1114 bytes} [Wed Aug 31 00:08:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:34.428291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77d7a3-5b50-48ee-bb01-a442b9e138cb None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:34.513785 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd298a4-0bdb-4178-bc03-4bbe60ba8fa3 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:34.537463 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77d7a3-5b50-48ee-bb01-a442b9e138cb None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:34.591694 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85897f-1c9c-431f-8977-0406ab412cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:34.592481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85897f-1c9c-431f-8977-0406ab412cc6 None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:34.592752 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85897f-1c9c-431f-8977-0406ab412cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:34.593022 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85897f-1c9c-431f-8977-0406ab412cc6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:34.593704 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85897f-1c9c-431f-8977-0406ab412cc6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:34.595089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85897f-1c9c-431f-8977-0406ab412cc6 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:34.600241 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:34.627113 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 466/920] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:08:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:08:34.642281 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd298a4-0bdb-4178-bc03-4bbe60ba8fa3 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:34.690136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad50fa6-b7a7-4b78-a0b1-792f4290850a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:34.690751 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad50fa6-b7a7-4b78-a0b1-792f4290850a None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:34.691020 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad50fa6-b7a7-4b78-a0b1-792f4290850a None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:34.691315 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad50fa6-b7a7-4b78-a0b1-792f4290850a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:34.691906 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad50fa6-b7a7-4b78-a0b1-792f4290850a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:34.693441 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad50fa6-b7a7-4b78-a0b1-792f4290850a None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:34.705369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-0ad50fa6-b7a7-4b78-a0b1-792f4290850a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:08:34.705369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:34.705369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:08:34.705369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:08:34.705369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:08:34.705369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:08:34.705369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:08:34.705369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:34.705369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:08:34.705369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:34.705369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:08:34.705369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:34.705369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:34.705369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:34.705369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:34.705369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:34.705369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:34.707879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:34.707879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:34.707879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:34.707879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:34.707879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:34.707879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:34.707879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:08:34.707879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:08:34.707879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:08:34.707879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:08:34.707879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:34.707879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:34.707879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:34.707879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:34.707879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:34.707879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:34.708972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:34.708972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:34.708972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:34.708972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:34.708972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:34.708972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:34.708972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:34.708972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:34.708972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:34.708972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:34.708972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:08:34.708972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:34.708972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:08:34.708972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:34.708972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 455/921] 10.222.0.67 () {56 vars in 1126 bytes} [Wed Aug 31 00:08:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:34.714135 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85897f-1c9c-431f-8977-0406ab412cc6 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:34.828856 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85897f-1c9c-431f-8977-0406ab412cc6 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:34.864217 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65de8d-09fd-4b23-9565-45e13b9861fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:34.864826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65de8d-09fd-4b23-9565-45e13b9861fc None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:34.865102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65de8d-09fd-4b23-9565-45e13b9861fc None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:34.865372 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65de8d-09fd-4b23-9565-45e13b9861fc None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:34.866001 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b65de8d-09fd-4b23-9565-45e13b9861fc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:34.866304 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b65de8d-09fd-4b23-9565-45e13b9861fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:34.867803 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b65de8d-09fd-4b23-9565-45e13b9861fc None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:34.873474 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 467/922] 10.222.0.67 () {56 vars in 1122 bytes} [Wed Aug 31 00:08:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:34.879393 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad50fa6-b7a7-4b78-a0b1-792f4290850a None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:34.922488 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:34.987580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad50fa6-b7a7-4b78-a0b1-792f4290850a None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:35.023710 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0455b65-6cd7-4a1b-b50c-45f15fd534c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:35.024414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0455b65-6cd7-4a1b-b50c-45f15fd534c7 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:35.024691 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0455b65-6cd7-4a1b-b50c-45f15fd534c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:35.024960 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0455b65-6cd7-4a1b-b50c-45f15fd534c7 None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:35.025597 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0455b65-6cd7-4a1b-b50c-45f15fd534c7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:35.025897 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0455b65-6cd7-4a1b-b50c-45f15fd534c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:35.027286 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0455b65-6cd7-4a1b-b50c-45f15fd534c7 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:35.033057 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 456/923] 10.222.0.67 () {56 vars in 1122 bytes} [Wed Aug 31 00:08:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:35.038502 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65de8d-09fd-4b23-9565-45e13b9861fc None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:35.104220 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:35.151995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65de8d-09fd-4b23-9565-45e13b9861fc None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:35.189114 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c902f-85a5-4b93-b999-bbeb19bf0197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:35.189725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c902f-85a5-4b93-b999-bbeb19bf0197 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:35.190003 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c902f-85a5-4b93-b999-bbeb19bf0197 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:35.190269 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c902f-85a5-4b93-b999-bbeb19bf0197 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:35.190969 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c902f-85a5-4b93-b999-bbeb19bf0197 None admin] RBAC: Adding query filter params (interface=public, service_id=726c88722e9949ccb0a432e4467250d7, region_id=RegionOne) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:35.191381 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c902f-85a5-4b93-b999-bbeb19bf0197 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:35.192803 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c902f-85a5-4b93-b999-bbeb19bf0197 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:35.198403 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 468/924] 10.222.0.67 () {56 vars in 1251 bytes} [Wed Aug 31 00:08:35 2022] GET /identity/v3/endpoints?service_id=726c88722e9949ccb0a432e4467250d7&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:35.203650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0455b65-6cd7-4a1b-b50c-45f15fd534c7 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:35.220816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7118c-9d6b-4162-8946-b6bd840fe179 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:35.221500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7118c-9d6b-4162-8946-b6bd840fe179 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:35.221792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7118c-9d6b-4162-8946-b6bd840fe179 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:35.223663 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 469/925] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:08:35.229312 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3988b0-8fc4-401d-8f4e-2a47504eb4e8 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:35.229731 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3988b0-8fc4-401d-8f4e-2a47504eb4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:35.231181 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3988b0-8fc4-401d-8f4e-2a47504eb4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:35.261436 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-eb3988b0-8fc4-401d-8f4e-2a47504eb4e8 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:35.263464 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-eb3988b0-8fc4-401d-8f4e-2a47504eb4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:35.318642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0455b65-6cd7-4a1b-b50c-45f15fd534c7 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:35.321104 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:35.357289 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 470/926] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:08:35.368607 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6efbef-6aeb-4149-b32c-bf6113068f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:35.369044 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6efbef-6aeb-4149-b32c-bf6113068f5b None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:35.369325 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6efbef-6aeb-4149-b32c-bf6113068f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:35.369595 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6efbef-6aeb-4149-b32c-bf6113068f5b None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:35.370205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6efbef-6aeb-4149-b32c-bf6113068f5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:35.371600 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6efbef-6aeb-4149-b32c-bf6113068f5b None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:35.378570 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 457/927] 10.222.0.67 () {56 vars in 1087 bytes} [Wed Aug 31 00:08:35 2022] GET /identity/v3/services => generated 1497 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:08:35.436622 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac8fcf-653a-4d9f-8d73-36fb6e668872 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:35.437174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac8fcf-653a-4d9f-8d73-36fb6e668872 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:35.437174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac8fcf-653a-4d9f-8d73-36fb6e668872 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:35.438888 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 471/928] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:35.443978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c79cf-6a06-4492-b2af-e1320c2b375a None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:35.444258 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c79cf-6a06-4492-b2af-e1320c2b375a None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:35.444528 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c79cf-6a06-4492-b2af-e1320c2b375a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:35.464634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-ea8c79cf-6a06-4492-b2af-e1320c2b375a None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:35.465987 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-ea8c79cf-6a06-4492-b2af-e1320c2b375a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:35.541602 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 458/929] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:35.639189 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac8fcf-653a-4d9f-8d73-36fb6e668872 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:35.692325 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:35.745745 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac8fcf-653a-4d9f-8d73-36fb6e668872 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:35.780983 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ea70f-697b-4a4d-83da-fae38d01e075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:35.781628 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ea70f-697b-4a4d-83da-fae38d01e075 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:35.781908 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ea70f-697b-4a4d-83da-fae38d01e075 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:35.782180 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ea70f-697b-4a4d-83da-fae38d01e075 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:35.782828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ea70f-697b-4a4d-83da-fae38d01e075 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:35.784238 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ea70f-697b-4a4d-83da-fae38d01e075 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:35.793367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-b31ea70f-697b-4a4d-83da-fae38d01e075 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 00:08:35.793367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:35.793367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:08:35.793367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:08:35.793367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:08:35.793367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:08:35.793367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:08:35.793367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:35.793367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 00:08:35.793367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:35.793367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:08:35.793367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:35.793367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:35.793367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:35.793367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:35.793367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:35.794433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:35.794433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:35.794433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:35.794433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:35.794433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:35.794433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:35.794433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:35.794433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:08:35.794433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:08:35.794433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:08:35.794433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:08:35.794433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:35.794433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:35.794433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:35.794433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:35.794433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:35.794433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:35.795798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:35.795798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:35.795798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:35.795798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:35.795798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:35.795798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:35.795798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:35.795798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:35.795798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:35.795798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:35.795798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:08:35.795798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:35.795798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 00:08:35.795798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:35.795798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 472/930] 10.222.0.67 () {56 vars in 1132 bytes} [Wed Aug 31 00:08:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:35.801122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c79cf-6a06-4492-b2af-e1320c2b375a None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:35.862806 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ea70f-697b-4a4d-83da-fae38d01e075 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:35.913159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:35.920703 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c79cf-6a06-4492-b2af-e1320c2b375a None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:35.966851 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08e72f-c5ee-4ff9-84e9-a9d928154922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:35.968222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08e72f-c5ee-4ff9-84e9-a9d928154922 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:35.968726 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08e72f-c5ee-4ff9-84e9-a9d928154922 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:35.969687 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08e72f-c5ee-4ff9-84e9-a9d928154922 None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:35.973296 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e08e72f-c5ee-4ff9-84e9-a9d928154922 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:35.974083 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e08e72f-c5ee-4ff9-84e9-a9d928154922 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:35.975995 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e08e72f-c5ee-4ff9-84e9-a9d928154922 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:35.983566 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 459/931] 10.222.0.67 () {56 vars in 1126 bytes} [Wed Aug 31 00:08:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:35.990266 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08e72f-c5ee-4ff9-84e9-a9d928154922 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:36.004491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ea70f-697b-4a4d-83da-fae38d01e075 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:36.022414 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:36.051346 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a310ebd-329e-49d5-9688-ce51349f7be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:36.051906 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a310ebd-329e-49d5-9688-ce51349f7be5 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:36.052242 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a310ebd-329e-49d5-9688-ce51349f7be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:36.052517 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a310ebd-329e-49d5-9688-ce51349f7be5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:36.066890 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a310ebd-329e-49d5-9688-ce51349f7be5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:08:36.074852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-2a310ebd-329e-49d5-9688-ce51349f7be5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 00:08:36.074852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:36.074852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:36.074852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:36.074852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:36.074852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:36.074852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:36.074852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:36.074852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:36.074852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:36.074852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:36.074852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:36.074852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:08:36.074852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:08:36.074852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:08:36.074852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:08:36.076869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:08:36.076869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:08:36.076869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:08:36.076869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:08:36.076869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:36.076869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:36.076869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:36.076869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:36.076869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:36.076869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:36.076869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:36.076869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:36.076869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:36.076869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:36.076869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:36.076869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:36.077746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:36.077746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:36.077746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:36.077746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:36.077746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:08:36.077746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:08:36.077746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:08:36.077746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:08:36.077746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:08:36.077746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:08:36.077746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 00:08:36.077746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:36.077746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 473/932] 10.222.0.67 () {56 vars in 1120 bytes} [Wed Aug 31 00:08:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:36.084727 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a310ebd-329e-49d5-9688-ce51349f7be5 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:36.126782 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08e72f-c5ee-4ff9-84e9-a9d928154922 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:36.165693 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cc9de-3c8a-4dff-b02f-ed453aa6013b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:36.166562 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cc9de-3c8a-4dff-b02f-ed453aa6013b None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:36.166677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cc9de-3c8a-4dff-b02f-ed453aa6013b None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:36.166948 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cc9de-3c8a-4dff-b02f-ed453aa6013b None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:36.167666 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16cc9de-3c8a-4dff-b02f-ed453aa6013b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:36.167971 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16cc9de-3c8a-4dff-b02f-ed453aa6013b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:36.170270 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16cc9de-3c8a-4dff-b02f-ed453aa6013b None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:36.180443 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 460/933] 10.222.0.67 () {56 vars in 1126 bytes} [Wed Aug 31 00:08:35 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:36.181408 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cc9de-3c8a-4dff-b02f-ed453aa6013b None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:36.195529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a310ebd-329e-49d5-9688-ce51349f7be5 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:36.210735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:36.233935 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ade408-e3f9-431d-939d-19f22169c9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:36.234996 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ade408-e3f9-431d-939d-19f22169c9ff None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:36.235501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ade408-e3f9-431d-939d-19f22169c9ff None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:36.235978 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ade408-e3f9-431d-939d-19f22169c9ff None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:36.236847 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ade408-e3f9-431d-939d-19f22169c9ff None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:36.237375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ade408-e3f9-431d-939d-19f22169c9ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:36.238964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ade408-e3f9-431d-939d-19f22169c9ff None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:36.253475 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 474/934] 10.222.0.67 () {56 vars in 1115 bytes} [Wed Aug 31 00:08:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:36.254677 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ade408-e3f9-431d-939d-19f22169c9ff None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:36.291829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:36.297504 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cc9de-3c8a-4dff-b02f-ed453aa6013b None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:36.339383 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f9714-79e0-4a7a-ba52-b07c01f91bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:36.340011 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f9714-79e0-4a7a-ba52-b07c01f91bc6 None admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:36.340265 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f9714-79e0-4a7a-ba52-b07c01f91bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:36.340542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f9714-79e0-4a7a-ba52-b07c01f91bc6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:36.341912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f9714-79e0-4a7a-ba52-b07c01f91bc6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:36.343396 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f9714-79e0-4a7a-ba52-b07c01f91bc6 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:36.372487 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ade408-e3f9-431d-939d-19f22169c9ff None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:36.375298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 461/935] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:08:36 2022] POST /identity/v3/endpoints => generated 347 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:08:36.423953 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591fbdbe-6538-47a1-befb-75741ab2a7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:36.425025 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591fbdbe-6538-47a1-befb-75741ab2a7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:36.425518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591fbdbe-6538-47a1-befb-75741ab2a7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:36.425994 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591fbdbe-6538-47a1-befb-75741ab2a7b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:36.439521 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591fbdbe-6538-47a1-befb-75741ab2a7b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:36.441800 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591fbdbe-6538-47a1-befb-75741ab2a7b3 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:36.459221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-591fbdbe-6538-47a1-befb-75741ab2a7b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:36.459221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:36.459221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:36.459221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:36.459221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:36.459221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:36.459221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:36.459221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:36.459221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:36.459221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:36.459221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:36.459221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:36.459221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:36.459221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:36.459221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:36.459221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:36.460225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:36.460225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:36.460225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:36.460225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:36.460225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:36.460225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:36.460225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:36.460225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:36.460225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:36.460225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:36.460225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:36.460225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:36.460225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:36.460225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:36.460225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:36.460225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:36.460225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:36.461143 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:36.461143 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:36.461143 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:36.467528 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 475/936] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:36.470456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f9714-79e0-4a7a-ba52-b07c01f91bc6 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:36.566641 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:36.600579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f9714-79e0-4a7a-ba52-b07c01f91bc6 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:36.642338 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f450302-2209-4f91-86fc-62540615a7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:36.643262 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f450302-2209-4f91-86fc-62540615a7f4 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:36.643755 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f450302-2209-4f91-86fc-62540615a7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:36.644191 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f450302-2209-4f91-86fc-62540615a7f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:36.644988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f450302-2209-4f91-86fc-62540615a7f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:36.645451 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f450302-2209-4f91-86fc-62540615a7f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:36.646972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f450302-2209-4f91-86fc-62540615a7f4 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:36.661080 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 462/937] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:36.667299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591fbdbe-6538-47a1-befb-75741ab2a7b3 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:36.756322 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465b187-47c9-46f8-8996-719afafbb112 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:36.756623 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465b187-47c9-46f8-8996-719afafbb112 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:36.756894 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465b187-47c9-46f8-8996-719afafbb112 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:36.759135 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 463/938] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:36.764471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0889aed-d696-41c7-ba41-d5f0a6f081e5 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:36.764753 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0889aed-d696-41c7-ba41-d5f0a6f081e5 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:36.765025 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0889aed-d696-41c7-ba41-d5f0a6f081e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:36.765778 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:36.784709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-b0889aed-d696-41c7-ba41-d5f0a6f081e5 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:36.785892 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b0889aed-d696-41c7-ba41-d5f0a6f081e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:36.786032 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591fbdbe-6538-47a1-befb-75741ab2a7b3 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:36.824454 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948b765-dfe6-4af5-9ead-7aa1adfdffd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:36.825234 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948b765-dfe6-4af5-9ead-7aa1adfdffd7 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:36.826217 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948b765-dfe6-4af5-9ead-7aa1adfdffd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:36.826801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948b765-dfe6-4af5-9ead-7aa1adfdffd7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:36.835344 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8948b765-dfe6-4af5-9ead-7aa1adfdffd7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:36.836930 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8948b765-dfe6-4af5-9ead-7aa1adfdffd7 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:36.851439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-8948b765-dfe6-4af5-9ead-7aa1adfdffd7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 00:08:36.851439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:36.851439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:36.851439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:36.851439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:36.851439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:36.851439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:36.851439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:36.851439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:36.851439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:36.851439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:36.851439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:36.851439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:08:36.851439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:08:36.851439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:08:36.851439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:08:36.852407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:36.852407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:36.852407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:08:36.852407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:36.852407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:08:36.852407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:36.852407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:36.852407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:36.852407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:36.852407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:36.852407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:36.852407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:36.852407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:36.852407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:36.852407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:36.852407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:36.852407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:36.852407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:36.853335 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:36.853335 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:36.853335 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:08:36.853335 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:08:36.853335 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:08:36.853335 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:08:36.853335 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:08:36.853335 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:08:36.853335 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 00:08:36.853335 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:36.853335 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 476/939] 10.222.0.67 () {56 vars in 1117 bytes} [Wed Aug 31 00:08:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:36.859962 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948b765-dfe6-4af5-9ead-7aa1adfdffd7 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:36.871128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 464/940] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:36.974983 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948b765-dfe6-4af5-9ead-7aa1adfdffd7 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:37.012382 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b24c3-20bf-4696-8a10-c220fdb4b6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:37.013847 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b24c3-20bf-4696-8a10-c220fdb4b6d1 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:37.014279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b24c3-20bf-4696-8a10-c220fdb4b6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:37.014611 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b24c3-20bf-4696-8a10-c220fdb4b6d1 None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:37.015385 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b24c3-20bf-4696-8a10-c220fdb4b6d1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:37.015696 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b24c3-20bf-4696-8a10-c220fdb4b6d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:37.017125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b24c3-20bf-4696-8a10-c220fdb4b6d1 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:37.032153 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:37.036839 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 477/941] 10.222.0.67 () {56 vars in 1149 bytes} [Wed Aug 31 00:08:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:37.042492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0889aed-d696-41c7-ba41-d5f0a6f081e5 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:37.124457 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:37.150754 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0889aed-d696-41c7-ba41-d5f0a6f081e5 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:37.172080 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b24c3-20bf-4696-8a10-c220fdb4b6d1 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:37.186613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b3f476-6639-40df-96dd-fddcd13f63fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:37.187277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3f476-6639-40df-96dd-fddcd13f63fb None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:37.187557 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3f476-6639-40df-96dd-fddcd13f63fb None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:37.187824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3f476-6639-40df-96dd-fddcd13f63fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:37.201343 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b3f476-6639-40df-96dd-fddcd13f63fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:37.202796 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b3f476-6639-40df-96dd-fddcd13f63fb None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:37.221978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-e0b3f476-6639-40df-96dd-fddcd13f63fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:37.221978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:37.221978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:37.221978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:37.221978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:37.221978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:37.221978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:37.221978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:37.221978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:37.221978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:37.221978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:37.221978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:37.221978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:37.221978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:37.221978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:37.221978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:37.222950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:37.222950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:37.222950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:37.222950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:37.222950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:37.222950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:37.222950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:37.222950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:37.222950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:37.222950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:37.222950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:37.222950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:37.222950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:37.222950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:37.222950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:37.222950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:37.222950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:37.224182 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:37.224182 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:37.224182 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:37.224182 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 465/942] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:37.229111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b3f476-6639-40df-96dd-fddcd13f63fb None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:37.300408 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b24c3-20bf-4696-8a10-c220fdb4b6d1 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:37.345202 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b3f476-6639-40df-96dd-fddcd13f63fb None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:37.346707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf0fb7-02b6-4271-b626-69071edbde14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:37.347358 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf0fb7-02b6-4271-b626-69071edbde14 None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:37.347640 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf0fb7-02b6-4271-b626-69071edbde14 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:37.347914 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf0fb7-02b6-4271-b626-69071edbde14 None admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:37.348841 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf0fb7-02b6-4271-b626-69071edbde14 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:37.351210 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf0fb7-02b6-4271-b626-69071edbde14 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:37.353814 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: WARNING keystone.server.flask.common [None req-8bbf0fb7-02b6-4271-b626-69071edbde14 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 31 00:08:37.372086 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:37.380497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.sql.core [None req-8bbf0fb7-02b6-4271-b626-69071edbde14 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 00:08:37.380497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 00:08:37.380497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [parameters: {'id': '301ecaf8ad6041c0adebfc754fcb1c2e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 00:08:37.380497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81472) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 00:08:37.388210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e7133-c628-4cc2-bacc-ae95cee399e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:37.389117 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e7133-c628-4cc2-bacc-ae95cee399e2 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:37.389619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e7133-c628-4cc2-bacc-ae95cee399e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:37.390102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e7133-c628-4cc2-bacc-ae95cee399e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:37.390988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e7133-c628-4cc2-bacc-ae95cee399e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:37.391530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e7133-c628-4cc2-bacc-ae95cee399e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:37.393210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e7133-c628-4cc2-bacc-ae95cee399e2 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:37.399970 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-8bbf0fb7-02b6-4271-b626-69071edbde14 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 301ecaf8ad6041c0adebfc754fcb1c2e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 301ecaf8ad6041c0adebfc754fcb1c2e. Aug 31 00:08:37.399970 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:37.399970 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 00:08:37.399970 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 00:08:37.399970 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 00:08:37.399970 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 00:08:37.399970 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 00:08:37.399970 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 00:08:37.399970 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 00:08:37.399970 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application conn.query(q) Aug 31 00:08:37.399970 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 00:08:37.399970 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 00:08:37.399970 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 00:08:37.400786 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application result.read() Aug 31 00:08:37.400786 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 00:08:37.400786 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 00:08:37.400786 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 00:08:37.400786 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 00:08:37.400786 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 00:08:37.400786 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 00:08:37.400786 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 00:08:37.400786 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 00:08:37.400786 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 00:08:37.400786 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:37.400786 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 00:08:37.400786 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:37.400786 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:37.400786 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 00:08:37.400786 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 00:08:37.400786 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 31 00:08:37.400786 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 31 00:08:37.407300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 00:08:37.407300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application next(self.gen) Aug 31 00:08:37.407300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 00:08:37.407300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application yield resource Aug 31 00:08:37.407300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 00:08:37.407300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application next(self.gen) Aug 31 00:08:37.407300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 00:08:37.407300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 00:08:37.407300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 00:08:37.407300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 00:08:37.407300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 00:08:37.407300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise self.value Aug 31 00:08:37.407300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 00:08:37.407300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 00:08:37.407300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 00:08:37.407300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application session.commit() Aug 31 00:08:37.407300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 00:08:37.408461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 00:08:37.408461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 00:08:37.408461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 00:08:37.408461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 00:08:37.408461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self.session.flush() Aug 31 00:08:37.408461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 00:08:37.408461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 00:08:37.408461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 00:08:37.408461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 00:08:37.408461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 00:08:37.408461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application compat.raise_( Aug 31 00:08:37.408461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 00:08:37.408461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception Aug 31 00:08:37.408461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 00:08:37.408461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 00:08:37.408461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 00:08:37.408461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 00:08:37.409353 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 00:08:37.409353 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 00:08:37.409353 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 00:08:37.409353 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 00:08:37.409353 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 00:08:37.409353 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 00:08:37.409353 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 00:08:37.409353 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 00:08:37.409353 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 00:08:37.409353 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 00:08:37.409353 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 00:08:37.409353 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 00:08:37.409353 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 00:08:37.409353 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 00:08:37.409353 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 00:08:37.410139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 00:08:37.410139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 00:08:37.410139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception Aug 31 00:08:37.410139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 00:08:37.410139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 00:08:37.410139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 00:08:37.410139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 00:08:37.410139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 00:08:37.410139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 00:08:37.410139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 00:08:37.410139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application conn.query(q) Aug 31 00:08:37.410139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 00:08:37.410139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 00:08:37.410139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 00:08:37.410139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application result.read() Aug 31 00:08:37.410139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 00:08:37.410139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 00:08:37.413145 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 00:08:37.413145 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 00:08:37.413145 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 00:08:37.413145 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 00:08:37.413145 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 00:08:37.413145 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 00:08:37.413145 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 00:08:37.413145 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 00:08:37.413145 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [parameters: {'id': '301ecaf8ad6041c0adebfc754fcb1c2e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 00:08:37.413145 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 00:08:37.413145 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:37.413145 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:08:37.413145 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:37.413145 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:37.413145 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 31 00:08:37.415627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 31 00:08:37.415627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 00:08:37.415627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 00:08:37.415627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 31 00:08:37.415627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:37.415627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:08:37.415627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:37.415627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:37.415627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:37.415627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:37.415627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:37.415627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:37.415627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:37.415627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:37.415627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:37.415627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:37.415627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:37.417893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:37.417893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 31 00:08:37.417893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 31 00:08:37.417893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:37.417893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:37.417893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 31 00:08:37.417893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 31 00:08:37.417893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 301ecaf8ad6041c0adebfc754fcb1c2e. Aug 31 00:08:37.417893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:37.417893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 478/943] 10.222.0.67 () {60 vars in 1140 bytes} [Wed Aug 31 00:08:37 2022] POST /identity/v3/projects => generated 262 bytes in 230 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:37.417893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee679121-3dab-4a60-863f-99dd819cc8d5 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:37.417893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee679121-3dab-4a60-863f-99dd819cc8d5 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:37.420164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee679121-3dab-4a60-863f-99dd819cc8d5 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:37.420345 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 466/944] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:37.420345 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e7133-c628-4cc2-bacc-ae95cee399e2 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:37.420626 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 479/945] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:37 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:08:37.420626 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee679121-3dab-4a60-863f-99dd819cc8d5 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:37.484811 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:37.521691 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e7133-c628-4cc2-bacc-ae95cee399e2 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:37.542079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee679121-3dab-4a60-863f-99dd819cc8d5 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:37.558706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4b099-2000-45f2-a6f5-c025e26c58f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:37.559374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4b099-2000-45f2-a6f5-c025e26c58f6 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:37.559677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4b099-2000-45f2-a6f5-c025e26c58f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:37.560136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4b099-2000-45f2-a6f5-c025e26c58f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:37.568507 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d4b099-2000-45f2-a6f5-c025e26c58f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:37.569999 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d4b099-2000-45f2-a6f5-c025e26c58f6 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:37.579939 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06201978-3429-488d-b87a-9f6b91a7f015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:37.580575 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06201978-3429-488d-b87a-9f6b91a7f015 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:37.580855 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06201978-3429-488d-b87a-9f6b91a7f015 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:37.581129 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06201978-3429-488d-b87a-9f6b91a7f015 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:37.581640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-72d4b099-2000-45f2-a6f5-c025e26c58f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:37.581640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:37.581640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:37.581640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:37.581640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:37.581640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:37.581640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:37.581640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:37.581640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:37.581640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:37.581640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:37.581640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:37.581640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:08:37.581640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:08:37.581640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:08:37.582492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:08:37.582492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:37.582492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:37.582492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:37.582492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:37.582492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:37.582492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:37.582492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:37.582492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:37.582492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:37.582492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:37.582492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:37.582492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:37.582492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:37.582492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:37.582492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:37.582492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:37.583413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:08:37.583413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:08:37.583413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:08:37.583413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:08:37.583413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:08:37.583413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:08:37.583413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:37.583413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:37.583413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 467/946] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:37.583964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a127aa3-6b96-4267-b43b-24fbd6793cc8 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:37.584540 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a127aa3-6b96-4267-b43b-24fbd6793cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:37.584540 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a127aa3-6b96-4267-b43b-24fbd6793cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:37.594315 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06201978-3429-488d-b87a-9f6b91a7f015 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:37.595909 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06201978-3429-488d-b87a-9f6b91a7f015 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:37.604644 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-2a127aa3-6b96-4267-b43b-24fbd6793cc8 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:37.605821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-2a127aa3-6b96-4267-b43b-24fbd6793cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:37.607780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-06201978-3429-488d-b87a-9f6b91a7f015 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:37.607780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:37.607780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:37.607780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:37.607780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:37.607780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:37.607780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:37.607780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:37.607780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:37.607780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:37.607780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:37.607780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:37.607780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:08:37.607780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:08:37.607780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:08:37.608671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:08:37.608671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:37.608671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:37.608671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:37.608671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:37.608671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:37.608671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:37.608671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:37.608671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:37.608671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:37.608671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:37.608671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:37.608671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:37.608671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:37.608671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:37.608671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:37.608671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:37.609588 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:08:37.609588 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:08:37.609588 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:08:37.609588 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:08:37.609588 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:08:37.609588 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:08:37.609588 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:37.609588 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:37.609588 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 480/947] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:37.610898 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06201978-3429-488d-b87a-9f6b91a7f015 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:37.659802 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:37.697543 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 468/948] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:37.698594 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a127aa3-6b96-4267-b43b-24fbd6793cc8 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:37.730970 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06201978-3429-488d-b87a-9f6b91a7f015 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:37.755824 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:37.777325 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf64083-a9f4-4643-8d0f-ccfb3a3ac7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:37.778144 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf64083-a9f4-4643-8d0f-ccfb3a3ac7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:37.778339 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf64083-a9f4-4643-8d0f-ccfb3a3ac7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:37.779346 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf64083-a9f4-4643-8d0f-ccfb3a3ac7b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:37.780237 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf64083-a9f4-4643-8d0f-ccfb3a3ac7b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:37.780547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf64083-a9f4-4643-8d0f-ccfb3a3ac7b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:37.782445 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf64083-a9f4-4643-8d0f-ccfb3a3ac7b8 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:37.807628 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 481/949] 10.222.0.67 () {56 vars in 1112 bytes} [Wed Aug 31 00:08:37 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:37.836234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a127aa3-6b96-4267-b43b-24fbd6793cc8 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:37.878475 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4864580-f52c-44aa-9c38-16dcc1077056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:37.879517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4864580-f52c-44aa-9c38-16dcc1077056 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:37.879941 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4864580-f52c-44aa-9c38-16dcc1077056 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:37.880356 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4864580-f52c-44aa-9c38-16dcc1077056 None admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:37.881241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4864580-f52c-44aa-9c38-16dcc1077056 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:37.881708 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4864580-f52c-44aa-9c38-16dcc1077056 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:37.883322 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4864580-f52c-44aa-9c38-16dcc1077056 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:37.902778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 469/950] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:08:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:37.904935 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf64083-a9f4-4643-8d0f-ccfb3a3ac7b8 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:37.985735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4864580-f52c-44aa-9c38-16dcc1077056 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:38.031055 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:38.044499 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf64083-a9f4-4643-8d0f-ccfb3a3ac7b8 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:38.087924 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd567dde-770d-413e-9ebc-5a9764571de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:38.087924 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd567dde-770d-413e-9ebc-5a9764571de5 None admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:38.088420 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd567dde-770d-413e-9ebc-5a9764571de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:38.088800 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd567dde-770d-413e-9ebc-5a9764571de5 None admin] PATH_INFO: `/v3/projects/cd1e6daca4264a0cbc11a9160d1213eb/users/a8fb7a9acefd4b418c992661037c2ed9/roles/39f496940d6844c8a43185fbd5fc1d21` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:38.092563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd567dde-770d-413e-9ebc-5a9764571de5 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd1e6daca4264a0cbc11a9160d1213eb, user_id=a8fb7a9acefd4b418c992661037c2ed9, role_id=39f496940d6844c8a43185fbd5fc1d21)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:38.094819 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd567dde-770d-413e-9ebc-5a9764571de5 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:38.119060 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 482/951] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 31 00:08:37 2022] PUT /identity/v3/projects/cd1e6daca4264a0cbc11a9160d1213eb/users/a8fb7a9acefd4b418c992661037c2ed9/roles/39f496940d6844c8a43185fbd5fc1d21 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:08:38.135000 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4864580-f52c-44aa-9c38-16dcc1077056 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:38.174912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba663e47-6acd-48a0-b075-b291487dbe4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:38.175604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba663e47-6acd-48a0-b075-b291487dbe4b None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:38.175944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba663e47-6acd-48a0-b075-b291487dbe4b None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:38.176218 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba663e47-6acd-48a0-b075-b291487dbe4b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:38.176819 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba663e47-6acd-48a0-b075-b291487dbe4b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:38.178215 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba663e47-6acd-48a0-b075-b291487dbe4b None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:38.189052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-ba663e47-6acd-48a0-b075-b291487dbe4b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:08:38.189052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:38.189052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:08:38.189052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:08:38.189052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:08:38.189052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:08:38.189052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:08:38.189052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:38.189052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:08:38.189052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:38.189052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:08:38.189052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:38.189052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:38.189052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:38.189052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:38.189052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:38.189052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:38.190122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:38.190122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:38.190122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:38.190122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:38.190122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:38.190122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:38.190122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:08:38.190122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:08:38.190122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:08:38.190122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:08:38.190122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:38.190122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:38.190122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:38.190122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:38.190122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:38.190122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:38.191764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:38.191764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:38.191764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:38.191764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:38.191764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:38.191764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:38.191764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:38.191764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:38.191764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:38.191764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:38.191764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:08:38.191764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:38.191764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:08:38.191764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:38.191764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 470/952] 10.222.0.67 () {56 vars in 1126 bytes} [Wed Aug 31 00:08:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:38.196402 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd567dde-770d-413e-9ebc-5a9764571de5 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:38.236219 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:38.319073 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd567dde-770d-413e-9ebc-5a9764571de5 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:38.355693 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48c0fa-6670-4367-8507-5634be1a598e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:38.356307 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48c0fa-6670-4367-8507-5634be1a598e None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:38.357123 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48c0fa-6670-4367-8507-5634be1a598e None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:38.357394 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48c0fa-6670-4367-8507-5634be1a598e None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:38.358469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a48c0fa-6670-4367-8507-5634be1a598e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:38.358773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a48c0fa-6670-4367-8507-5634be1a598e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:38.360179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a48c0fa-6670-4367-8507-5634be1a598e None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:38.365758 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 483/953] 10.222.0.67 () {56 vars in 1122 bytes} [Wed Aug 31 00:08:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:38.371488 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba663e47-6acd-48a0-b075-b291487dbe4b None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:38.477855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:38.484531 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba663e47-6acd-48a0-b075-b291487dbe4b None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:38.504364 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb3f5d-40e5-4207-8f08-2e7946d0b373 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:38.504869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb3f5d-40e5-4207-8f08-2e7946d0b373 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:38.504949 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb3f5d-40e5-4207-8f08-2e7946d0b373 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:38.506335 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 484/954] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:08:38.512535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab224f-faa5-479b-9e68-5a143d60dc8b None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:38.513663 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab224f-faa5-479b-9e68-5a143d60dc8b None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:38.513776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab224f-faa5-479b-9e68-5a143d60dc8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:38.528282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a95916-a88d-49ed-aef8-2cb08f06bc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:38.529417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a95916-a88d-49ed-aef8-2cb08f06bc69 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:38.529929 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a95916-a88d-49ed-aef8-2cb08f06bc69 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:38.530401 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a95916-a88d-49ed-aef8-2cb08f06bc69 None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:38.531260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a95916-a88d-49ed-aef8-2cb08f06bc69 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:38.531781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a95916-a88d-49ed-aef8-2cb08f06bc69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:38.532410 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-43ab224f-faa5-479b-9e68-5a143d60dc8b None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:38.533582 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a95916-a88d-49ed-aef8-2cb08f06bc69 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:38.533934 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-43ab224f-faa5-479b-9e68-5a143d60dc8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:38.540946 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 471/955] 10.222.0.67 () {56 vars in 1122 bytes} [Wed Aug 31 00:08:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:38.547085 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a95916-a88d-49ed-aef8-2cb08f06bc69 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:38.615309 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 485/956] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:08:38.655964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a95916-a88d-49ed-aef8-2cb08f06bc69 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:38.695082 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a54dd1f-bfcb-4a55-9242-18d9351ec46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:38.695725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54dd1f-bfcb-4a55-9242-18d9351ec46b None admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:38.696076 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54dd1f-bfcb-4a55-9242-18d9351ec46b None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:38.696354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54dd1f-bfcb-4a55-9242-18d9351ec46b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:38.697044 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a54dd1f-bfcb-4a55-9242-18d9351ec46b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:38.698438 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a54dd1f-bfcb-4a55-9242-18d9351ec46b None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:38.724151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 472/957] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:08:38 2022] POST /identity/v3/endpoints => generated 349 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:08:38.984762 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab224f-faa5-479b-9e68-5a143d60dc8b None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:39.018037 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:39.093554 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab224f-faa5-479b-9e68-5a143d60dc8b None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:39.134769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda178c6-1e20-4606-a3c2-b0a0782fdf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:39.135437 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda178c6-1e20-4606-a3c2-b0a0782fdf7b None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:39.135720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda178c6-1e20-4606-a3c2-b0a0782fdf7b None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:39.135994 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda178c6-1e20-4606-a3c2-b0a0782fdf7b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:39.136578 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda178c6-1e20-4606-a3c2-b0a0782fdf7b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:39.138621 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda178c6-1e20-4606-a3c2-b0a0782fdf7b None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:39.147789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-cda178c6-1e20-4606-a3c2-b0a0782fdf7b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 00:08:39.147789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:39.147789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:08:39.147789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:08:39.147789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:08:39.147789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:08:39.147789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:08:39.147789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:39.147789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 00:08:39.147789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:39.147789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:08:39.147789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:39.147789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:39.147789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:39.147789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:39.147789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:39.147789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:39.148863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:39.148863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:39.148863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:39.148863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:39.148863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:39.148863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:39.148863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:08:39.148863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:08:39.148863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:08:39.148863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:08:39.148863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:39.148863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:39.148863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:39.148863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:39.148863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:39.148863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:39.150330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:39.150330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:39.150330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:39.150330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:39.150330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:39.150330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:39.150330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:39.150330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:39.150330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:39.150330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:39.150330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:08:39.150330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:39.150330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 00:08:39.150330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:39.150330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 486/958] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:39.155166 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a54dd1f-bfcb-4a55-9242-18d9351ec46b None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:39.175427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:39.260738 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a54dd1f-bfcb-4a55-9242-18d9351ec46b None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:39.296056 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49655d7a-4d83-4a65-8beb-ab71d1e8fcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:39.296674 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49655d7a-4d83-4a65-8beb-ab71d1e8fcd6 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:39.296962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49655d7a-4d83-4a65-8beb-ab71d1e8fcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:39.297236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49655d7a-4d83-4a65-8beb-ab71d1e8fcd6 None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:39.297907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49655d7a-4d83-4a65-8beb-ab71d1e8fcd6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:39.298216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49655d7a-4d83-4a65-8beb-ab71d1e8fcd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:39.299630 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49655d7a-4d83-4a65-8beb-ab71d1e8fcd6 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:39.305173 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 473/959] 10.222.0.67 () {56 vars in 1112 bytes} [Wed Aug 31 00:08:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:39.311316 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda178c6-1e20-4606-a3c2-b0a0782fdf7b None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:39.410969 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:39.421915 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda178c6-1e20-4606-a3c2-b0a0782fdf7b None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:39.457801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf58c5b-383a-473b-b797-81df9198f57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:39.458419 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf58c5b-383a-473b-b797-81df9198f57d None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:39.458701 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf58c5b-383a-473b-b797-81df9198f57d None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:39.458976 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf58c5b-383a-473b-b797-81df9198f57d None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:39.459649 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf58c5b-383a-473b-b797-81df9198f57d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:39.459957 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf58c5b-383a-473b-b797-81df9198f57d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:39.461344 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf58c5b-383a-473b-b797-81df9198f57d None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:39.466755 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 487/960] 10.222.0.67 () {56 vars in 1112 bytes} [Wed Aug 31 00:08:39 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:39.953127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada2e44-3236-4af6-89d3-40a6d66e4396 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:39.953647 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada2e44-3236-4af6-89d3-40a6d66e4396 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:39.953755 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada2e44-3236-4af6-89d3-40a6d66e4396 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:39.955321 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 474/961] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:39.960625 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8ce70-27a0-4429-9d64-db6d1d8247d7 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:39.961030 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8ce70-27a0-4429-9d64-db6d1d8247d7 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:39.961217 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8ce70-27a0-4429-9d64-db6d1d8247d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:39.986162 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-0ad8ce70-27a0-4429-9d64-db6d1d8247d7 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:39.987403 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-0ad8ce70-27a0-4429-9d64-db6d1d8247d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:40.067762 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 488/962] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:40.281027 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df1bd9-1836-4770-a97c-06eb34b3ca0c None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:40.281322 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df1bd9-1836-4770-a97c-06eb34b3ca0c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:40.281506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df1bd9-1836-4770-a97c-06eb34b3ca0c None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:40.283128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 475/963] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:40.288386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4a726-a49c-46ac-ad08-4ecd5d73cd8d None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:40.288898 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4a726-a49c-46ac-ad08-4ecd5d73cd8d None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:40.289372 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4a726-a49c-46ac-ad08-4ecd5d73cd8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:40.308695 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-88a4a726-a49c-46ac-ad08-4ecd5d73cd8d None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:40.310985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-88a4a726-a49c-46ac-ad08-4ecd5d73cd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:40.353524 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df1bd9-1836-4770-a97c-06eb34b3ca0c None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:40.383818 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 489/964] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:40.444454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:40.465132 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df1bd9-1836-4770-a97c-06eb34b3ca0c None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:40.502002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eaa8c7-b887-4fb1-a85c-30c9bbd2c838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:40.502627 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eaa8c7-b887-4fb1-a85c-30c9bbd2c838 None admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:40.502908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eaa8c7-b887-4fb1-a85c-30c9bbd2c838 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:40.503208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eaa8c7-b887-4fb1-a85c-30c9bbd2c838 None admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:40.503873 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eaa8c7-b887-4fb1-a85c-30c9bbd2c838 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:40.505310 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eaa8c7-b887-4fb1-a85c-30c9bbd2c838 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:40.507887 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: WARNING keystone.server.flask.common [None req-43eaa8c7-b887-4fb1-a85c-30c9bbd2c838 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 31 00:08:40.523084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.sql.core [None req-43eaa8c7-b887-4fb1-a85c-30c9bbd2c838 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 00:08:40.523084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 00:08:40.523084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [parameters: {'user_id': '2d59ec555d8b4431abcef685352df139', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 00:08:40.523084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81471) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 00:08:40.531330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-43eaa8c7-b887-4fb1-a85c-30c9bbd2c838 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 00:08:40.531330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:40.531330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 00:08:40.531330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 00:08:40.531330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 00:08:40.531330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 00:08:40.531330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 00:08:40.531330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 00:08:40.531330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 00:08:40.531330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application conn.query(q) Aug 31 00:08:40.531330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 00:08:40.531330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 00:08:40.531330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 00:08:40.531330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application result.read() Aug 31 00:08:40.531330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 00:08:40.532196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 00:08:40.532196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 00:08:40.532196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 00:08:40.532196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 00:08:40.532196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 00:08:40.532196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 00:08:40.532196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 00:08:40.532196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 00:08:40.532196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:40.532196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 00:08:40.532196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:40.532196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:40.532196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 00:08:40.532196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 00:08:40.532196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 31 00:08:40.532196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 31 00:08:40.532196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 00:08:40.532196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application next(self.gen) Aug 31 00:08:40.533750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 00:08:40.533750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application yield resource Aug 31 00:08:40.533750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 00:08:40.533750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application next(self.gen) Aug 31 00:08:40.533750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 00:08:40.533750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 00:08:40.533750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 00:08:40.533750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 00:08:40.533750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 00:08:40.533750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise self.value Aug 31 00:08:40.533750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 00:08:40.533750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 00:08:40.533750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 00:08:40.533750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application session.commit() Aug 31 00:08:40.533750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 00:08:40.533750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 00:08:40.533750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 00:08:40.534650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 00:08:40.534650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 00:08:40.534650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self.session.flush() Aug 31 00:08:40.534650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 00:08:40.534650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 00:08:40.534650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 00:08:40.534650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 00:08:40.534650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 00:08:40.534650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application compat.raise_( Aug 31 00:08:40.534650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 00:08:40.534650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception Aug 31 00:08:40.534650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 00:08:40.534650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 00:08:40.534650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 00:08:40.534650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 00:08:40.534650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 00:08:40.534650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 00:08:40.536529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 00:08:40.536529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 00:08:40.536529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 31 00:08:40.536529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 31 00:08:40.536529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 00:08:40.536529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 00:08:40.536529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 00:08:40.536529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 00:08:40.536529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 00:08:40.536529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 00:08:40.536529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 00:08:40.536529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 00:08:40.536529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 00:08:40.536529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 00:08:40.536529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 00:08:40.536529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception Aug 31 00:08:40.538124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 00:08:40.538124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 00:08:40.538124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 00:08:40.538124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 00:08:40.538124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 00:08:40.538124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 00:08:40.538124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 00:08:40.538124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application conn.query(q) Aug 31 00:08:40.538124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 00:08:40.538124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 00:08:40.538124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 00:08:40.538124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application result.read() Aug 31 00:08:40.538124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 00:08:40.538124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 00:08:40.538124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 00:08:40.538124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 00:08:40.539190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 00:08:40.539190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 00:08:40.539190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 00:08:40.539190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 00:08:40.539190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 00:08:40.539190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 00:08:40.539190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [parameters: {'user_id': '2d59ec555d8b4431abcef685352df139', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 00:08:40.539190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 00:08:40.539190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:40.539190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:08:40.539190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:40.539190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:40.539190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:40.539190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:40.539190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:40.540216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:40.540216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:40.540216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:40.540216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:40.540216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:40.540216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:40.540216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:40.540216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 31 00:08:40.540216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 31 00:08:40.540216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:40.540216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:40.540216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:08:40.540216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:40.540216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:08:40.540216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:40.540216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 31 00:08:40.540216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 31 00:08:40.541277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 31 00:08:40.541277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 31 00:08:40.541277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 00:08:40.541277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 00:08:40.541277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 00:08:40.541277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:40.541277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 476/965] 10.222.0.67 () {60 vars in 1131 bytes} [Wed Aug 31 00:08:40 2022] POST /identity/v3/users => generated 160 bytes in 180 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:40.541741 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a4a726-a49c-46ac-ad08-4ecd5d73cd8d None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:40.558436 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:40.655388 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a4a726-a49c-46ac-ad08-4ecd5d73cd8d None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:40.679224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eaa8c7-b887-4fb1-a85c-30c9bbd2c838 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:40.696805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f06bf8-f34c-43d8-aa4e-de201d16c881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:40.697618 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f06bf8-f34c-43d8-aa4e-de201d16c881 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:40.697901 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f06bf8-f34c-43d8-aa4e-de201d16c881 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:40.698178 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f06bf8-f34c-43d8-aa4e-de201d16c881 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:40.708188 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f06bf8-f34c-43d8-aa4e-de201d16c881 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:40.710139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f06bf8-f34c-43d8-aa4e-de201d16c881 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:40.727179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-32f06bf8-f34c-43d8-aa4e-de201d16c881 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 00:08:40.727179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:40.727179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:40.727179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:40.727179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:40.727179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:40.727179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:40.727179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:40.727179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:40.727179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:40.727179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:40.727179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:40.727179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:08:40.727179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:08:40.727179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:08:40.727179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:08:40.728329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:40.728329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:40.728329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:08:40.728329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:40.728329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:08:40.728329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:40.728329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:40.728329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:40.728329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:40.728329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:40.728329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:40.728329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:40.728329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:40.728329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:40.728329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:40.728329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:40.728329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:40.728329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:40.729540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:40.729540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:40.729540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:08:40.729540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:08:40.729540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:08:40.729540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:08:40.729540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:08:40.729540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:08:40.729540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 00:08:40.729540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:40.729540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 490/966] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:08:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:40.730400 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed77cd6-58f8-4b23-b185-c9f0964fc458 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:40.730483 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed77cd6-58f8-4b23-b185-c9f0964fc458 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:40.730723 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed77cd6-58f8-4b23-b185-c9f0964fc458 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:40.739319 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 491/967] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:40.739319 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed77cd6-58f8-4b23-b185-c9f0964fc458 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:40.808399 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eaa8c7-b887-4fb1-a85c-30c9bbd2c838 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:40.823731 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:40.845398 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253d9ac-8f96-410a-8b75-fde9b511f26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:40.846036 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253d9ac-8f96-410a-8b75-fde9b511f26d None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:40.846319 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253d9ac-8f96-410a-8b75-fde9b511f26d None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:40.846593 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253d9ac-8f96-410a-8b75-fde9b511f26d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:40.851137 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed77cd6-58f8-4b23-b185-c9f0964fc458 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:40.858315 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253d9ac-8f96-410a-8b75-fde9b511f26d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:08:40.863728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-9253d9ac-8f96-410a-8b75-fde9b511f26d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 00:08:40.863728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:40.863728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:40.863728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:40.863728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:40.863728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:40.863728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:40.863728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:40.863728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:40.863728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:40.863728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:40.863728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:40.863728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:08:40.863728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:08:40.863728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:08:40.863728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:08:40.864890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:08:40.864890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:08:40.864890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:08:40.864890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:08:40.864890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:40.864890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:40.864890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:40.864890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:40.864890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:40.864890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:40.864890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:40.864890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:40.864890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:40.864890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:40.864890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:40.864890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:40.865898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:40.865898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:40.865898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:40.865898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:40.865898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:08:40.865898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:08:40.865898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:08:40.865898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:08:40.865898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:08:40.865898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:08:40.865898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 00:08:40.865898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:40.865898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 477/968] 10.222.0.67 () {56 vars in 1120 bytes} [Wed Aug 31 00:08:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:40.866790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351ef37-056c-4388-bd99-7edc8e60aff0 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:40.867059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351ef37-056c-4388-bd99-7edc8e60aff0 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:40.867302 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351ef37-056c-4388-bd99-7edc8e60aff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:40.874641 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:40.887537 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-c351ef37-056c-4388-bd99-7edc8e60aff0 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:40.889996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c351ef37-056c-4388-bd99-7edc8e60aff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:40.896460 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eacca65-a8c0-4b70-9877-81cfce820d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:40.897215 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eacca65-a8c0-4b70-9877-81cfce820d53 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:40.897607 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eacca65-a8c0-4b70-9877-81cfce820d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:40.898374 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eacca65-a8c0-4b70-9877-81cfce820d53 None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:40.899224 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eacca65-a8c0-4b70-9877-81cfce820d53 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:40.899644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eacca65-a8c0-4b70-9877-81cfce820d53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:40.901828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eacca65-a8c0-4b70-9877-81cfce820d53 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:40.941141 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 492/969] 10.222.0.67 () {56 vars in 1101 bytes} [Wed Aug 31 00:08:40 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:40.942366 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eacca65-a8c0-4b70-9877-81cfce820d53 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:40.987633 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 478/970] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:41.070239 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eacca65-a8c0-4b70-9877-81cfce820d53 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:41.103072 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:41.109273 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721f957-d125-4219-ab1f-feb10d49d229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:41.110987 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721f957-d125-4219-ab1f-feb10d49d229 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:41.111288 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721f957-d125-4219-ab1f-feb10d49d229 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:41.111555 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721f957-d125-4219-ab1f-feb10d49d229 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:41.112224 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c721f957-d125-4219-ab1f-feb10d49d229 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:41.112522 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c721f957-d125-4219-ab1f-feb10d49d229 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:41.113881 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c721f957-d125-4219-ab1f-feb10d49d229 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:41.123156 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 493/971] 10.222.0.67 () {56 vars in 1115 bytes} [Wed Aug 31 00:08:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:41.129239 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c351ef37-056c-4388-bd99-7edc8e60aff0 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:41.234169 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c351ef37-056c-4388-bd99-7edc8e60aff0 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:41.265064 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:41.271356 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa2c1b-824c-4f1b-a36b-5cadb63cb0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:41.272078 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa2c1b-824c-4f1b-a36b-5cadb63cb0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:41.272354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa2c1b-824c-4f1b-a36b-5cadb63cb0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:41.272623 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa2c1b-824c-4f1b-a36b-5cadb63cb0f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:41.276230 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721f957-d125-4219-ab1f-feb10d49d229 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:41.284111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa2c1b-824c-4f1b-a36b-5cadb63cb0f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:41.286200 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa2c1b-824c-4f1b-a36b-5cadb63cb0f4 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:41.302901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-b3aa2c1b-824c-4f1b-a36b-5cadb63cb0f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:41.302901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:41.302901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:41.302901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:41.302901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:41.302901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:41.302901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:41.302901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:41.302901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:41.302901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:41.302901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:41.302901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:41.302901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:41.302901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:41.302901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:41.302901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:41.309535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:41.309535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:41.309535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:41.309535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:41.309535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:41.309535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:41.309535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:41.309535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:41.309535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:41.309535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:41.309535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:41.309535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:41.309535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:41.309535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:41.309535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:41.309535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:41.309535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:41.311413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:41.311413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:41.311413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:41.311413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 479/972] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:41.311413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa2c1b-824c-4f1b-a36b-5cadb63cb0f4 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:41.402319 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721f957-d125-4219-ab1f-feb10d49d229 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:41.435532 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa2c1b-824c-4f1b-a36b-5cadb63cb0f4 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:41.438809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130969e-7450-46cd-9095-0fb185f4adf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:41.439467 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130969e-7450-46cd-9095-0fb185f4adf2 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:41.439743 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130969e-7450-46cd-9095-0fb185f4adf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:41.440090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130969e-7450-46cd-9095-0fb185f4adf2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:41.440673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130969e-7450-46cd-9095-0fb185f4adf2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:41.442066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130969e-7450-46cd-9095-0fb185f4adf2 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:41.451549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-1130969e-7450-46cd-9095-0fb185f4adf2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:08:41.451549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:41.451549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:08:41.451549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:08:41.451549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:08:41.451549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:08:41.451549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:08:41.451549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:41.451549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:08:41.451549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:41.451549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:08:41.451549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:41.451549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:41.451549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:41.451549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:41.451549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:41.451549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:41.452542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:41.452542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:41.452542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:41.452542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:41.452542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:41.452542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:41.452542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:08:41.452542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:08:41.452542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:08:41.452542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:08:41.452542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:41.452542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:41.452542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:41.452542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:41.452542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:41.452542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:41.454440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:41.454440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:41.454440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:41.454440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:41.454440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:41.454440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:41.454440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:41.454440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:41.454440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:41.454440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:41.454440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:08:41.454440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:41.454440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:08:41.454440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:41.454440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 494/973] 10.222.0.67 () {56 vars in 1126 bytes} [Wed Aug 31 00:08:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:41.459980 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130969e-7450-46cd-9095-0fb185f4adf2 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:41.474565 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457460c-907a-425b-b510-0325dfd076d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:41.475209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457460c-907a-425b-b510-0325dfd076d8 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:41.475490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457460c-907a-425b-b510-0325dfd076d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:41.475763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457460c-907a-425b-b510-0325dfd076d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:41.476708 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457460c-907a-425b-b510-0325dfd076d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:41.476708 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457460c-907a-425b-b510-0325dfd076d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:41.478042 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457460c-907a-425b-b510-0325dfd076d8 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:41.498881 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 480/974] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:41.500454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457460c-907a-425b-b510-0325dfd076d8 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:41.528748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:41.548138 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:41.571488 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130969e-7450-46cd-9095-0fb185f4adf2 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:41.602905 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457460c-907a-425b-b510-0325dfd076d8 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:41.605966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15440d25-7dc5-4db4-9bbf-9afc33bc9a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:41.606622 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15440d25-7dc5-4db4-9bbf-9afc33bc9a28 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:41.606900 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15440d25-7dc5-4db4-9bbf-9afc33bc9a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:41.607201 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15440d25-7dc5-4db4-9bbf-9afc33bc9a28 None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:41.607824 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15440d25-7dc5-4db4-9bbf-9afc33bc9a28 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:41.608121 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15440d25-7dc5-4db4-9bbf-9afc33bc9a28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:41.609475 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15440d25-7dc5-4db4-9bbf-9afc33bc9a28 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:41.614160 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 495/975] 10.222.0.67 () {56 vars in 1122 bytes} [Wed Aug 31 00:08:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:41.615607 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b4375-8d8d-49d9-982e-e4eb7007076f None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:41.615893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b4375-8d8d-49d9-982e-e4eb7007076f None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:41.616163 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b4375-8d8d-49d9-982e-e4eb7007076f None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:41.618043 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 496/976] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:41.620039 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b4375-8d8d-49d9-982e-e4eb7007076f None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:41.639930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167b182-081c-4641-832f-a3aab5e15842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:41.640585 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167b182-081c-4641-832f-a3aab5e15842 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:41.640868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167b182-081c-4641-832f-a3aab5e15842 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:41.641133 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167b182-081c-4641-832f-a3aab5e15842 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:41.648162 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c167b182-081c-4641-832f-a3aab5e15842 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:41.649600 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c167b182-081c-4641-832f-a3aab5e15842 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:41.662363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-c167b182-081c-4641-832f-a3aab5e15842 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 00:08:41.662363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:41.662363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:41.662363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:41.662363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:41.662363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:41.662363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:41.662363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:41.662363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:41.662363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:41.662363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:41.662363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:41.662363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:08:41.662363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:08:41.662363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:08:41.662363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:08:41.668672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:41.668672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:41.668672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:08:41.668672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:41.668672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:08:41.668672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:41.668672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:41.668672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:41.668672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:41.668672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:41.668672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:41.668672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:41.668672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:41.668672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:41.668672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:41.668672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:41.668672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:41.668672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:41.670893 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:41.670893 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:41.670893 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:08:41.670893 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:08:41.670893 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:08:41.670893 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:08:41.670893 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:08:41.670893 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:08:41.670893 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 00:08:41.670893 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:41.670893 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 481/977] 10.222.0.67 () {56 vars in 1117 bytes} [Wed Aug 31 00:08:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:41.670893 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ec762-c95f-43b6-9103-5d74a7b46f73 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:41.670893 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ec762-c95f-43b6-9103-5d74a7b46f73 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:41.672463 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ec762-c95f-43b6-9103-5d74a7b46f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:41.684358 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-120ec762-c95f-43b6-9103-5d74a7b46f73 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:41.686445 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-120ec762-c95f-43b6-9103-5d74a7b46f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:41.724582 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b4375-8d8d-49d9-982e-e4eb7007076f None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:41.746658 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:41.754862 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 482/978] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:41.755738 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ec762-c95f-43b6-9103-5d74a7b46f73 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:41.772017 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac908b42-534a-4064-b5c0-fd29250cd697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:41.773046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac908b42-534a-4064-b5c0-fd29250cd697 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:41.773509 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac908b42-534a-4064-b5c0-fd29250cd697 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:41.773794 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac908b42-534a-4064-b5c0-fd29250cd697 None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:41.774435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac908b42-534a-4064-b5c0-fd29250cd697 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:41.774744 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac908b42-534a-4064-b5c0-fd29250cd697 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:41.776158 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac908b42-534a-4064-b5c0-fd29250cd697 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:41.786604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 497/979] 10.222.0.67 () {56 vars in 1122 bytes} [Wed Aug 31 00:08:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:41.787670 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac908b42-534a-4064-b5c0-fd29250cd697 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:41.799581 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:41.874449 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ec762-c95f-43b6-9103-5d74a7b46f73 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:41.894697 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac908b42-534a-4064-b5c0-fd29250cd697 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:41.911480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8bdb91-0907-4169-8454-5828e3929178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:41.912094 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bdb91-0907-4169-8454-5828e3929178 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:41.912432 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bdb91-0907-4169-8454-5828e3929178 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:41.912649 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bdb91-0907-4169-8454-5828e3929178 None admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:41.913392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8bdb91-0907-4169-8454-5828e3929178 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:41.913700 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8bdb91-0907-4169-8454-5828e3929178 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:41.915145 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8bdb91-0907-4169-8454-5828e3929178 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:41.933071 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 483/980] 10.222.0.67 () {56 vars in 1149 bytes} [Wed Aug 31 00:08:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:41.934118 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ecb4a-03ea-4f21-ab75-5c4a47202f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:41.935315 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ecb4a-03ea-4f21-ab75-5c4a47202f6a None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:41.935800 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ecb4a-03ea-4f21-ab75-5c4a47202f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:41.936203 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ecb4a-03ea-4f21-ab75-5c4a47202f6a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:41.936950 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ecb4a-03ea-4f21-ab75-5c4a47202f6a None admin] RBAC: Adding query filter params (interface=admin, service_id=726c88722e9949ccb0a432e4467250d7, region_id=RegionOne) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:41.937289 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ecb4a-03ea-4f21-ab75-5c4a47202f6a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:41.938928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ecb4a-03ea-4f21-ab75-5c4a47202f6a None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:41.939489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8bdb91-0907-4169-8454-5828e3929178 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:41.949451 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 498/981] 10.222.0.67 () {56 vars in 1249 bytes} [Wed Aug 31 00:08:41 2022] GET /identity/v3/endpoints?service_id=726c88722e9949ccb0a432e4467250d7&interface=admin®ion_id=RegionOne => generated 193 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:41.951352 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ecb4a-03ea-4f21-ab75-5c4a47202f6a None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:42.007353 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:42.049740 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:42.059065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8bdb91-0907-4169-8454-5828e3929178 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:42.061738 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ecb4a-03ea-4f21-ab75-5c4a47202f6a None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:42.094097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a3781-3ab4-461c-a211-6b6340c24ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:42.094706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a3781-3ab4-461c-a211-6b6340c24ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:42.095049 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a3781-3ab4-461c-a211-6b6340c24ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:42.095286 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a3781-3ab4-461c-a211-6b6340c24ad6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:42.096205 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152559b-e55d-42a0-aa93-787c1768537c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:42.096814 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152559b-e55d-42a0-aa93-787c1768537c None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:42.097099 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152559b-e55d-42a0-aa93-787c1768537c None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:42.097370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152559b-e55d-42a0-aa93-787c1768537c None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:42.097976 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d152559b-e55d-42a0-aa93-787c1768537c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:42.099361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d152559b-e55d-42a0-aa93-787c1768537c None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:42.105291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 499/982] 10.222.0.67 () {56 vars in 1087 bytes} [Wed Aug 31 00:08:41 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:08:42.105878 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152559b-e55d-42a0-aa93-787c1768537c None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:42.177092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a3781-3ab4-461c-a211-6b6340c24ad6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:42.179121 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a3781-3ab4-461c-a211-6b6340c24ad6 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:42.213520 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152559b-e55d-42a0-aa93-787c1768537c None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:42.228082 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:42.252489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01aee35-ed17-47a7-b006-6d9606672d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:42.253089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01aee35-ed17-47a7-b006-6d9606672d12 None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:42.253374 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01aee35-ed17-47a7-b006-6d9606672d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:42.253647 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01aee35-ed17-47a7-b006-6d9606672d12 None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:42.254294 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01aee35-ed17-47a7-b006-6d9606672d12 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:42.255689 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01aee35-ed17-47a7-b006-6d9606672d12 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:42.275210 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 500/983] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 31 00:08:42 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:08:42.288090 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:42.300338 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 484/984] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:08:41 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:08:42.306666 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01aee35-ed17-47a7-b006-6d9606672d12 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:42.427088 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01aee35-ed17-47a7-b006-6d9606672d12 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:42.462526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de062ec3-4d1e-42a3-919a-c12be32324d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:42.463109 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de062ec3-4d1e-42a3-919a-c12be32324d1 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:42.463395 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de062ec3-4d1e-42a3-919a-c12be32324d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:42.463668 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de062ec3-4d1e-42a3-919a-c12be32324d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:42.475266 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de062ec3-4d1e-42a3-919a-c12be32324d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:42.476718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de062ec3-4d1e-42a3-919a-c12be32324d1 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:42.491163 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-de062ec3-4d1e-42a3-919a-c12be32324d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:42.491163 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:42.491163 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:42.491163 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:42.491163 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:42.491163 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:42.491163 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:42.491163 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:42.491163 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:42.491163 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:42.491163 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:42.491163 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:42.491163 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:42.491163 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:42.491163 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:42.491163 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:42.492102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:42.492102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:42.492102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:42.492102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:42.492102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:42.492102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:42.492102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:42.492102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:42.492102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:42.492102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:42.492102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:42.492102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:42.492102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:42.492102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:42.492102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:42.492102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:42.492102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:42.492987 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:42.492987 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:42.492987 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:42.492987 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 501/985] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:42.498397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a3781-3ab4-461c-a211-6b6340c24ad6 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:42.601498 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a3781-3ab4-461c-a211-6b6340c24ad6 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:42.636232 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a818b69-c0d0-43d4-80c9-d9fd49d16093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:42.636836 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a818b69-c0d0-43d4-80c9-d9fd49d16093 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:42.637110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a818b69-c0d0-43d4-80c9-d9fd49d16093 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:42.637375 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a818b69-c0d0-43d4-80c9-d9fd49d16093 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:42.638287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a818b69-c0d0-43d4-80c9-d9fd49d16093 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:42.638538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a818b69-c0d0-43d4-80c9-d9fd49d16093 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:42.640022 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a818b69-c0d0-43d4-80c9-d9fd49d16093 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:42.651655 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 485/986] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:42.657689 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de062ec3-4d1e-42a3-919a-c12be32324d1 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:42.691503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:42.779130 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de062ec3-4d1e-42a3-919a-c12be32324d1 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:42.813740 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e91325-1a96-4e03-a6c2-358add0377be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:42.814376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e91325-1a96-4e03-a6c2-358add0377be None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:42.814656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e91325-1a96-4e03-a6c2-358add0377be None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:42.814928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e91325-1a96-4e03-a6c2-358add0377be None admin] PATH_INFO: `/v3/projects/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:42.821532 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e91325-1a96-4e03-a6c2-358add0377be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:42.822976 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e91325-1a96-4e03-a6c2-358add0377be None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:42.834832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-23e91325-1a96-4e03-a6c2-358add0377be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:42.834832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:42.834832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:42.834832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:42.834832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:42.834832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:42.834832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:42.834832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:42.834832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:42.834832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:42.834832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:42.834832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:42.834832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:08:42.834832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:08:42.834832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:08:42.835952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:08:42.835952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:42.835952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:42.835952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:42.835952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:42.835952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:42.835952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:42.835952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:42.835952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:42.835952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:42.835952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:42.835952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:42.835952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:42.835952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:42.835952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:42.835952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:42.835952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:42.836849 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:08:42.836849 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:08:42.836849 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:08:42.836849 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:08:42.836849 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:08:42.836849 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:08:42.836849 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:42.836849 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:42.839339 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 502/987] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:42.842229 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a818b69-c0d0-43d4-80c9-d9fd49d16093 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:42.895741 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:42.944121 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a818b69-c0d0-43d4-80c9-d9fd49d16093 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:42.982961 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ea23d-05c2-41e2-adf5-c6dfa85cfec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:42.983606 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ea23d-05c2-41e2-adf5-c6dfa85cfec1 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:42.983882 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ea23d-05c2-41e2-adf5-c6dfa85cfec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:42.984151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ea23d-05c2-41e2-adf5-c6dfa85cfec1 None admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:42.984844 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ea23d-05c2-41e2-adf5-c6dfa85cfec1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:42.985144 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ea23d-05c2-41e2-adf5-c6dfa85cfec1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:42.986563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ea23d-05c2-41e2-adf5-c6dfa85cfec1 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:43.005995 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 486/988] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:08:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:43.007301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e91325-1a96-4e03-a6c2-358add0377be None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:43.101808 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:43.121752 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e91325-1a96-4e03-a6c2-358add0377be None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:43.158426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f35d2bc-cc2e-415e-a9af-259100ea65c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:43.159068 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35d2bc-cc2e-415e-a9af-259100ea65c9 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:43.159363 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35d2bc-cc2e-415e-a9af-259100ea65c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:43.159639 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35d2bc-cc2e-415e-a9af-259100ea65c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:43.160371 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f35d2bc-cc2e-415e-a9af-259100ea65c9 None admin] RBAC: Adding query filter params (role.id=39f496940d6844c8a43185fbd5fc1d21, scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb, user.id=a8fb7a9acefd4b418c992661037c2ed9) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:43.160684 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f35d2bc-cc2e-415e-a9af-259100ea65c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:43.162122 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f35d2bc-cc2e-415e-a9af-259100ea65c9 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:43.168222 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 503/989] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:08:43 2022] GET /identity/v3/role_assignments?user.id=a8fb7a9acefd4b418c992661037c2ed9&scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb&role.id=39f496940d6844c8a43185fbd5fc1d21 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:44.086161 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ec171-2db9-4a0e-b991-8acc9cc5afdf None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:44.086677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ec171-2db9-4a0e-b991-8acc9cc5afdf None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:44.086785 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ec171-2db9-4a0e-b991-8acc9cc5afdf None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:44.088108 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 487/990] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:44.093730 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd474a-00ae-4ca9-b6fe-f61b2b83a873 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:44.094019 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd474a-00ae-4ca9-b6fe-f61b2b83a873 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:44.094296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd474a-00ae-4ca9-b6fe-f61b2b83a873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:44.113324 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-23bd474a-00ae-4ca9-b6fe-f61b2b83a873 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:44.114505 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-23bd474a-00ae-4ca9-b6fe-f61b2b83a873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:44.183468 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 504/991] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:44 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:44.260437 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032287f-e979-412b-80c6-75bf76f1d6b2 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:44.260728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032287f-e979-412b-80c6-75bf76f1d6b2 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:44.260925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032287f-e979-412b-80c6-75bf76f1d6b2 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:44.262406 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 488/992] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:44.268331 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dbeb9-d5ef-40a3-844a-e246f435f768 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:44.268627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dbeb9-d5ef-40a3-844a-e246f435f768 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:44.268904 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dbeb9-d5ef-40a3-844a-e246f435f768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:44.289088 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-0c0dbeb9-d5ef-40a3-844a-e246f435f768 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:44.290430 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-0c0dbeb9-d5ef-40a3-844a-e246f435f768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:44.365676 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 505/993] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:44 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:44.458733 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032287f-e979-412b-80c6-75bf76f1d6b2 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:44.562937 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032287f-e979-412b-80c6-75bf76f1d6b2 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:44.577133 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:44.599642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b75074-8b05-4c39-bc1d-116a16e07f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:44.600277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b75074-8b05-4c39-bc1d-116a16e07f72 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:44.600554 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b75074-8b05-4c39-bc1d-116a16e07f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:44.600825 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b75074-8b05-4c39-bc1d-116a16e07f72 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:44.601409 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b75074-8b05-4c39-bc1d-116a16e07f72 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:44.602767 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b75074-8b05-4c39-bc1d-116a16e07f72 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:44.611718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-b4b75074-8b05-4c39-bc1d-116a16e07f72 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:08:44.611718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:44.611718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:08:44.611718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:08:44.611718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:08:44.611718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:08:44.611718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:08:44.611718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:44.611718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:08:44.611718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:44.611718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:08:44.611718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:44.611718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:44.611718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:44.611718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:44.611718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:44.611718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:44.612706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:44.612706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:44.612706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:44.612706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:44.612706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:44.612706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:44.612706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:08:44.612706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:08:44.612706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:08:44.612706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:08:44.612706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:44.612706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:44.612706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:44.612706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:44.612706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:44.612706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:44.613573 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:44.613573 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:44.613573 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:44.613573 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:44.613573 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:44.613573 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:44.613573 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:44.613573 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:44.613573 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:44.613573 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:44.613573 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:08:44.613573 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:44.613573 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 00:08:44.613573 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:44.613573 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 489/994] 10.222.0.67 () {56 vars in 1126 bytes} [Wed Aug 31 00:08:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:44.618639 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0dbeb9-d5ef-40a3-844a-e246f435f768 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:44.627737 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:44.636025 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b75074-8b05-4c39-bc1d-116a16e07f72 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:44.730704 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0dbeb9-d5ef-40a3-844a-e246f435f768 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:44.761164 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b75074-8b05-4c39-bc1d-116a16e07f72 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:44.768125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a045b-c3d7-4d2d-9940-457b37a0fa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:44.768746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a045b-c3d7-4d2d-9940-457b37a0fa4a None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:44.769029 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a045b-c3d7-4d2d-9940-457b37a0fa4a None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:44.769303 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a045b-c3d7-4d2d-9940-457b37a0fa4a None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:44.769937 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a045b-c3d7-4d2d-9940-457b37a0fa4a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:44.770241 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a045b-c3d7-4d2d-9940-457b37a0fa4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:44.771648 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a045b-c3d7-4d2d-9940-457b37a0fa4a None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:44.779123 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 506/995] 10.222.0.67 () {56 vars in 1122 bytes} [Wed Aug 31 00:08:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:44.784569 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a045b-c3d7-4d2d-9940-457b37a0fa4a None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:44.801688 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89374f-71f4-4e43-b43b-25854b126e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:44.802321 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89374f-71f4-4e43-b43b-25854b126e5f None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:44.802605 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89374f-71f4-4e43-b43b-25854b126e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:44.802878 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89374f-71f4-4e43-b43b-25854b126e5f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:44.803506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a89374f-71f4-4e43-b43b-25854b126e5f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:44.804867 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a89374f-71f4-4e43-b43b-25854b126e5f None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:44.814131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-9a89374f-71f4-4e43-b43b-25854b126e5f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 00:08:44.814131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:44.814131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:08:44.814131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:08:44.814131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:08:44.814131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:08:44.814131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:08:44.814131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:44.814131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 00:08:44.814131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:44.814131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:08:44.814131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:44.814131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:44.814131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:44.814131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:44.814131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:44.814131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:44.816165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:44.816165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:44.816165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:44.816165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:44.816165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:44.816165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:44.816165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:08:44.816165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:08:44.816165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:08:44.816165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:08:44.816165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:44.816165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:44.816165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:44.816165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:44.816165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:44.816165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:44.817857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:44.817857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:44.817857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:44.817857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:44.817857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:44.817857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:44.817857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:44.817857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:44.817857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:44.817857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:44.817857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:08:44.817857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:44.817857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 00:08:44.817857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:44.817857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 490/996] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:44.821128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89374f-71f4-4e43-b43b-25854b126e5f None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:44.866497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:44.871480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:44.907912 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a045b-c3d7-4d2d-9940-457b37a0fa4a None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:44.928997 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89374f-71f4-4e43-b43b-25854b126e5f None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:44.948081 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a5d34-fefb-4997-805d-3e0d22959062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:44.948695 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a5d34-fefb-4997-805d-3e0d22959062 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:44.948972 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a5d34-fefb-4997-805d-3e0d22959062 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:44.949247 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a5d34-fefb-4997-805d-3e0d22959062 None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:44.949877 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79a5d34-fefb-4997-805d-3e0d22959062 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:44.950181 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79a5d34-fefb-4997-805d-3e0d22959062 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:44.952503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79a5d34-fefb-4997-805d-3e0d22959062 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:44.957467 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 507/997] 10.222.0.67 () {56 vars in 1122 bytes} [Wed Aug 31 00:08:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:44.963008 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a5d34-fefb-4997-805d-3e0d22959062 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:44.965525 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be9bc8-0974-4cbc-9854-12179a9beeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:44.966716 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be9bc8-0974-4cbc-9854-12179a9beeab None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:44.967144 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be9bc8-0974-4cbc-9854-12179a9beeab None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:44.967473 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be9bc8-0974-4cbc-9854-12179a9beeab None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:44.968178 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be9bc8-0974-4cbc-9854-12179a9beeab None admin] RBAC: Adding query filter params (name=compute) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:44.968583 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be9bc8-0974-4cbc-9854-12179a9beeab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:44.970061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be9bc8-0974-4cbc-9854-12179a9beeab None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:44.979859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 491/998] 10.222.0.67 () {56 vars in 1112 bytes} [Wed Aug 31 00:08:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:44.980910 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be9bc8-0974-4cbc-9854-12179a9beeab None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:45.088587 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a5d34-fefb-4997-805d-3e0d22959062 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:45.094614 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be9bc8-0974-4cbc-9854-12179a9beeab None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:45.097502 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:45.128182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8ce82-861c-4893-9d5e-5a2368bb13cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:45.128789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8ce82-861c-4893-9d5e-5a2368bb13cb None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:45.129072 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8ce82-861c-4893-9d5e-5a2368bb13cb None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:45.129349 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8ce82-861c-4893-9d5e-5a2368bb13cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:45.130049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8ce82-861c-4893-9d5e-5a2368bb13cb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:45.131464 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8ce82-861c-4893-9d5e-5a2368bb13cb None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:45.133417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2eb83f-9fa6-40d9-9045-b00682264069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:45.134133 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2eb83f-9fa6-40d9-9045-b00682264069 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:45.134413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2eb83f-9fa6-40d9-9045-b00682264069 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:45.134777 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2eb83f-9fa6-40d9-9045-b00682264069 None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:45.135440 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2eb83f-9fa6-40d9-9045-b00682264069 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:45.135746 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2eb83f-9fa6-40d9-9045-b00682264069 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:45.136329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:45.137097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2eb83f-9fa6-40d9-9045-b00682264069 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:45.143429 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 492/999] 10.222.0.67 () {56 vars in 1112 bytes} [Wed Aug 31 00:08:44 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:45.149650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2eb83f-9fa6-40d9-9045-b00682264069 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:45.159075 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 508/1000] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:08:44 2022] POST /identity/v3/endpoints => generated 325 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:08:45.163610 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47fbe9-3e21-4939-bc72-165e7f81c808 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:45.163923 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47fbe9-3e21-4939-bc72-165e7f81c808 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:45.164165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47fbe9-3e21-4939-bc72-165e7f81c808 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:45.166218 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 509/1001] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:08:45.172336 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c299cf2-0494-4b9e-b995-4699117c8cd8 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:45.172647 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c299cf2-0494-4b9e-b995-4699117c8cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:45.172890 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c299cf2-0494-4b9e-b995-4699117c8cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:45.192377 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-3c299cf2-0494-4b9e-b995-4699117c8cd8 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:45.193597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-3c299cf2-0494-4b9e-b995-4699117c8cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:45.277693 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2eb83f-9fa6-40d9-9045-b00682264069 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:45.286727 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 510/1002] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:45 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:08:45.323200 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa881c02-1462-4ab3-8c7b-db8df937ff3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:45.323818 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa881c02-1462-4ab3-8c7b-db8df937ff3c None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:45.324095 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa881c02-1462-4ab3-8c7b-db8df937ff3c None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:45.324366 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa881c02-1462-4ab3-8c7b-db8df937ff3c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:45.325067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa881c02-1462-4ab3-8c7b-db8df937ff3c None admin] RBAC: Adding query filter params (interface=public, service_id=071e402a2aee4a82b35ad22b0fd7e745, region_id=RegionOne) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:45.325507 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa881c02-1462-4ab3-8c7b-db8df937ff3c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:45.326865 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa881c02-1462-4ab3-8c7b-db8df937ff3c None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:45.332717 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 493/1003] 10.222.0.67 () {56 vars in 1251 bytes} [Wed Aug 31 00:08:45 2022] GET /identity/v3/endpoints?service_id=071e402a2aee4a82b35ad22b0fd7e745&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:45.338282 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c299cf2-0494-4b9e-b995-4699117c8cd8 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:45.446990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c299cf2-0494-4b9e-b995-4699117c8cd8 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:45.469918 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:45.489036 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327fc5a5-badd-4718-bb7f-2cf21e80aa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:45.489649 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fc5a5-badd-4718-bb7f-2cf21e80aa0a None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:45.489926 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fc5a5-badd-4718-bb7f-2cf21e80aa0a None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:45.490196 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fc5a5-badd-4718-bb7f-2cf21e80aa0a None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:45.490802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327fc5a5-badd-4718-bb7f-2cf21e80aa0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:45.492258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327fc5a5-badd-4718-bb7f-2cf21e80aa0a None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:45.500830 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 511/1004] 10.222.0.67 () {56 vars in 1087 bytes} [Wed Aug 31 00:08:45 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:08:45.625799 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa881c02-1462-4ab3-8c7b-db8df937ff3c None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:45.669886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:45.760508 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa881c02-1462-4ab3-8c7b-db8df937ff3c None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:45.799234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1788b93-5daf-4cec-8111-ecf50d529fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:45.799874 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1788b93-5daf-4cec-8111-ecf50d529fca None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:45.800151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1788b93-5daf-4cec-8111-ecf50d529fca None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:45.800423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1788b93-5daf-4cec-8111-ecf50d529fca None admin] PATH_INFO: `/v3/services/image` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:45.801039 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1788b93-5daf-4cec-8111-ecf50d529fca None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:45.802408 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1788b93-5daf-4cec-8111-ecf50d529fca None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:45.812381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-d1788b93-5daf-4cec-8111-ecf50d529fca None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 00:08:45.812381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:45.812381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:08:45.812381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:08:45.812381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:08:45.812381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:08:45.812381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:08:45.812381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:45.812381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 00:08:45.812381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:45.812381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:08:45.812381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:45.812381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:45.812381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:45.812381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:45.812381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:45.812381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:45.814113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:45.814113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:45.814113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:45.814113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:45.814113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:45.814113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:45.814113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:08:45.814113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:08:45.814113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:08:45.814113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:08:45.814113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:45.814113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:45.814113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:45.814113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:45.814113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:45.814113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:45.815868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:45.815868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:45.815868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:45.815868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:45.815868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:45.815868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:45.815868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:45.815868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:45.815868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:45.815868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:45.815868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:08:45.815868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:45.815868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 00:08:45.815868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:45.815868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 494/1005] 10.222.0.67 () {56 vars in 1105 bytes} [Wed Aug 31 00:08:45 2022] GET /identity/v3/services/image => generated 86 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:45.821280 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327fc5a5-badd-4718-bb7f-2cf21e80aa0a None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:45.944773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327fc5a5-badd-4718-bb7f-2cf21e80aa0a None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:46.003358 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d014d88-03e4-4d50-9082-7126b7130c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:46.004368 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d014d88-03e4-4d50-9082-7126b7130c4d None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:46.004863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d014d88-03e4-4d50-9082-7126b7130c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:46.005358 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d014d88-03e4-4d50-9082-7126b7130c4d None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:46.006420 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d014d88-03e4-4d50-9082-7126b7130c4d None admin] RBAC: Adding query filter params (name=image) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:46.006826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d014d88-03e4-4d50-9082-7126b7130c4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:46.008905 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d014d88-03e4-4d50-9082-7126b7130c4d None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:46.022204 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 512/1006] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:46.024401 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1788b93-5daf-4cec-8111-ecf50d529fca None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:46.178452 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:46.187128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1788b93-5daf-4cec-8111-ecf50d529fca None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:46.226159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f12ec-5607-4212-bb56-24a897b3e08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:46.227282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f12ec-5607-4212-bb56-24a897b3e08e None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:46.227810 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f12ec-5607-4212-bb56-24a897b3e08e None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:46.228323 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f12ec-5607-4212-bb56-24a897b3e08e None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:46.229520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f12ec-5607-4212-bb56-24a897b3e08e None admin] RBAC: Adding query filter params (type=image) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:46.230093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f12ec-5607-4212-bb56-24a897b3e08e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:46.232355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f12ec-5607-4212-bb56-24a897b3e08e None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:46.237680 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 495/1007] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:46 2022] GET /identity/v3/services?type=image => generated 122 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:47.201185 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b45c3-47ba-44b9-89e4-6094f2590e90 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:47.201700 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b45c3-47ba-44b9-89e4-6094f2590e90 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:47.201777 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b45c3-47ba-44b9-89e4-6094f2590e90 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:47.203247 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 513/1008] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:47.208389 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c61c8-7f24-4936-aa52-1e573bc024e5 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:47.208779 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c61c8-7f24-4936-aa52-1e573bc024e5 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:47.208940 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c61c8-7f24-4936-aa52-1e573bc024e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:47.227651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-0a5c61c8-7f24-4936-aa52-1e573bc024e5 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:47.228833 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-0a5c61c8-7f24-4936-aa52-1e573bc024e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:47.299695 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 496/1009] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:47 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:47.520197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ae497-d83a-474d-92b2-2b01c5305480 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:47.520566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ae497-d83a-474d-92b2-2b01c5305480 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:47.520748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ae497-d83a-474d-92b2-2b01c5305480 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:47.522127 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 514/1010] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:47.527292 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9d2d4-68d1-437a-a8d0-7c857ff5682b None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:47.527584 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9d2d4-68d1-437a-a8d0-7c857ff5682b None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:47.527859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9d2d4-68d1-437a-a8d0-7c857ff5682b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:47.547086 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-51b9d2d4-68d1-437a-a8d0-7c857ff5682b None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:47.548275 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-51b9d2d4-68d1-437a-a8d0-7c857ff5682b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:47.572898 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ae497-d83a-474d-92b2-2b01c5305480 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:47.623709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 497/1011] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:47 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:47.630809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:47.685872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ae497-d83a-474d-92b2-2b01c5305480 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:47.720777 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35845ea5-2232-4f29-a7a8-c5fdbaa0320f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:47.721510 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35845ea5-2232-4f29-a7a8-c5fdbaa0320f None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:47.721787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35845ea5-2232-4f29-a7a8-c5fdbaa0320f None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:47.722055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35845ea5-2232-4f29-a7a8-c5fdbaa0320f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:47.723192 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35845ea5-2232-4f29-a7a8-c5fdbaa0320f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:47.724650 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35845ea5-2232-4f29-a7a8-c5fdbaa0320f None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:47.726364 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 515/1012] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:47.732671 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9d2d4-68d1-437a-a8d0-7c857ff5682b None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:47.756000 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:47.838499 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9d2d4-68d1-437a-a8d0-7c857ff5682b None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:47.875008 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4cd850-d7d2-4ec2-8b4f-4b64792e2079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:47.875656 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cd850-d7d2-4ec2-8b4f-4b64792e2079 None admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:47.875934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cd850-d7d2-4ec2-8b4f-4b64792e2079 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:47.876203 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cd850-d7d2-4ec2-8b4f-4b64792e2079 None admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:47.877116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4cd850-d7d2-4ec2-8b4f-4b64792e2079 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:47.878532 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4cd850-d7d2-4ec2-8b4f-4b64792e2079 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:47.898815 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35845ea5-2232-4f29-a7a8-c5fdbaa0320f None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:47.908771 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 498/1013] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 31 00:08:47 2022] POST /identity/v3/projects => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:08:48.021840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35845ea5-2232-4f29-a7a8-c5fdbaa0320f None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:48.040820 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:48.062902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a1b33e-d56c-4745-a7d5-e55b3700194c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:48.064222 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1b33e-d56c-4745-a7d5-e55b3700194c None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:48.064222 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1b33e-d56c-4745-a7d5-e55b3700194c None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:48.064222 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1b33e-d56c-4745-a7d5-e55b3700194c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:48.064674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a1b33e-d56c-4745-a7d5-e55b3700194c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:48.066067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a1b33e-d56c-4745-a7d5-e55b3700194c None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:48.075013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-f0a1b33e-d56c-4745-a7d5-e55b3700194c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 00:08:48.075013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:48.075013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:08:48.075013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:08:48.075013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:08:48.075013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:08:48.075013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:08:48.075013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:48.075013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 00:08:48.075013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:48.075013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:08:48.075013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:48.075013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:48.075013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:48.075013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:48.075013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:48.075013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:48.076728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:48.076728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:48.076728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:48.076728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:48.076728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:48.076728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:48.076728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:08:48.076728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:08:48.076728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:08:48.076728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:08:48.076728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:48.076728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:48.076728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:48.076728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:48.076728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:48.076728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:48.078165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:48.078165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:48.078165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:48.078165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:48.078165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:48.078165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:48.078165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:48.078165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:48.078165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:48.078165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:48.078165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:08:48.078165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:48.078165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 00:08:48.078165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:48.078165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 516/1014] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:48.083210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4cd850-d7d2-4ec2-8b4f-4b64792e2079 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:48.143800 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd7e73-4e10-4ca7-ac47-d1e41b456a79 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:48.144003 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd7e73-4e10-4ca7-ac47-d1e41b456a79 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:48.144275 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd7e73-4e10-4ca7-ac47-d1e41b456a79 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:48.145756 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 517/1015] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:08:48.151474 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1fe3e-dcd0-49f0-b12c-7e9cdecbf2f2 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:48.151474 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1fe3e-dcd0-49f0-b12c-7e9cdecbf2f2 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:48.151474 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1fe3e-dcd0-49f0-b12c-7e9cdecbf2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:48.168733 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:48.169494 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-a9c1fe3e-dcd0-49f0-b12c-7e9cdecbf2f2 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:48.170670 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a9c1fe3e-dcd0-49f0-b12c-7e9cdecbf2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:48.209025 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4cd850-d7d2-4ec2-8b4f-4b64792e2079 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:48.242929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 518/1016] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:48 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:08:48.245057 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d095fa-64aa-46e0-9045-ad3d19eb4b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:48.245667 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d095fa-64aa-46e0-9045-ad3d19eb4b59 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:48.245942 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d095fa-64aa-46e0-9045-ad3d19eb4b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:48.246212 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d095fa-64aa-46e0-9045-ad3d19eb4b59 None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:48.246846 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d095fa-64aa-46e0-9045-ad3d19eb4b59 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:48.247173 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d095fa-64aa-46e0-9045-ad3d19eb4b59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:48.249084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d095fa-64aa-46e0-9045-ad3d19eb4b59 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:48.254665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 499/1017] 10.222.0.67 () {56 vars in 1112 bytes} [Wed Aug 31 00:08:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:48.260255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c1fe3e-dcd0-49f0-b12c-7e9cdecbf2f2 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:48.363529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c1fe3e-dcd0-49f0-b12c-7e9cdecbf2f2 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:48.375260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:48.400379 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da36356-9ff3-4492-a454-8c3731c1f2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:48.400915 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da36356-9ff3-4492-a454-8c3731c1f2ef None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:48.401199 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da36356-9ff3-4492-a454-8c3731c1f2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:48.401474 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da36356-9ff3-4492-a454-8c3731c1f2ef None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:48.402106 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da36356-9ff3-4492-a454-8c3731c1f2ef None admin] RBAC: Adding query filter params (type=compute) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:48.402468 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da36356-9ff3-4492-a454-8c3731c1f2ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:48.404053 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da36356-9ff3-4492-a454-8c3731c1f2ef None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:48.413629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 519/1018] 10.222.0.67 () {56 vars in 1112 bytes} [Wed Aug 31 00:08:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:48.414895 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d095fa-64aa-46e0-9045-ad3d19eb4b59 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:48.513878 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da36356-9ff3-4492-a454-8c3731c1f2ef None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:48.516926 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d095fa-64aa-46e0-9045-ad3d19eb4b59 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:48.553352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5dff3-17f4-4d1b-8284-720235ce3448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:48.553956 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5dff3-17f4-4d1b-8284-720235ce3448 None admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:48.554238 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5dff3-17f4-4d1b-8284-720235ce3448 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:48.554509 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5dff3-17f4-4d1b-8284-720235ce3448 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:48.555199 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5dff3-17f4-4d1b-8284-720235ce3448 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:48.556750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5dff3-17f4-4d1b-8284-720235ce3448 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:48.564050 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:48.584382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 500/1019] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:08:48 2022] POST /identity/v3/endpoints => generated 334 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:08:48.632887 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da36356-9ff3-4492-a454-8c3731c1f2ef None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:48.679136 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985c735-3226-4bfa-947e-8bda6e27fac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:48.679663 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985c735-3226-4bfa-947e-8bda6e27fac7 None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:48.679942 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985c735-3226-4bfa-947e-8bda6e27fac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:48.680214 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985c735-3226-4bfa-947e-8bda6e27fac7 None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:48.681339 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985c735-3226-4bfa-947e-8bda6e27fac7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:48.682725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985c735-3226-4bfa-947e-8bda6e27fac7 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:48.703096 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 520/1020] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 31 00:08:48 2022] POST /identity/v3/services => generated 242 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:08:49.911017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1532d1c-d473-41a6-b6d7-cfa63211e556 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:49.911609 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1532d1c-d473-41a6-b6d7-cfa63211e556 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:49.911609 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1532d1c-d473-41a6-b6d7-cfa63211e556 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:49.913105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 501/1021] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:49.917990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53215702-36ba-413b-a4ec-8f66ebd91791 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:49.918373 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53215702-36ba-413b-a4ec-8f66ebd91791 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:49.918536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53215702-36ba-413b-a4ec-8f66ebd91791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:49.938718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-53215702-36ba-413b-a4ec-8f66ebd91791 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:49.939931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-53215702-36ba-413b-a4ec-8f66ebd91791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:49.964519 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:50.023877 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 521/1022] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:50.297611 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1532d1c-d473-41a6-b6d7-cfa63211e556 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:50.401702 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1532d1c-d473-41a6-b6d7-cfa63211e556 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:50.436837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61029eb2-79f0-45e6-9468-790f22d44c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:50.437478 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61029eb2-79f0-45e6-9468-790f22d44c64 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:50.437762 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61029eb2-79f0-45e6-9468-790f22d44c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:50.438036 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61029eb2-79f0-45e6-9468-790f22d44c64 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:50.439190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61029eb2-79f0-45e6-9468-790f22d44c64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:50.440628 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61029eb2-79f0-45e6-9468-790f22d44c64 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:50.442432 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 502/1023] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:50.448052 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53215702-36ba-413b-a4ec-8f66ebd91791 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:50.552104 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53215702-36ba-413b-a4ec-8f66ebd91791 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:50.587362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195618d-7203-4cf7-bb8f-3f8031c7f67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:50.587840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195618d-7203-4cf7-bb8f-3f8031c7f67a None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:50.588122 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195618d-7203-4cf7-bb8f-3f8031c7f67a None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:50.588396 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195618d-7203-4cf7-bb8f-3f8031c7f67a None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:50.589062 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195618d-7203-4cf7-bb8f-3f8031c7f67a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:50.590520 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195618d-7203-4cf7-bb8f-3f8031c7f67a None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:50.600011 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c7fd4-72e8-415e-aef1-e86e3518544b None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:50.600300 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c7fd4-72e8-415e-aef1-e86e3518544b None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:50.600582 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c7fd4-72e8-415e-aef1-e86e3518544b None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:50.602174 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 503/1024] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:50.607726 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa9d19-674d-480e-a845-75e9bac0b988 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:50.608097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa9d19-674d-480e-a845-75e9bac0b988 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:50.608372 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa9d19-674d-480e-a845-75e9bac0b988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:50.614073 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 522/1025] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 31 00:08:50 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:08:50.629401 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-c5aa9d19-674d-480e-a845-75e9bac0b988 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:50.630890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c5aa9d19-674d-480e-a845-75e9bac0b988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:50.670916 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3009b6-ddee-461a-93e9-f2d0ab5cac8d None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:50.671272 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3009b6-ddee-461a-93e9-f2d0ab5cac8d None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:50.671500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3009b6-ddee-461a-93e9-f2d0ab5cac8d None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:50.673023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 523/1026] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:08:50.678418 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075f4ff-d761-4cf0-a3b4-849d0204161c None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:50.678703 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075f4ff-d761-4cf0-a3b4-849d0204161c None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:50.678976 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075f4ff-d761-4cf0-a3b4-849d0204161c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:50.698997 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-6075f4ff-d761-4cf0-a3b4-849d0204161c None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:50.699924 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:50.700279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-6075f4ff-d761-4cf0-a3b4-849d0204161c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:50.709861 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:50.716810 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 504/1027] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:50 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:08:50.784068 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 524/1028] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:50 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:08:50.990542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa9d19-674d-480e-a845-75e9bac0b988 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:51.057429 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075f4ff-d761-4cf0-a3b4-849d0204161c None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:51.095676 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa9d19-674d-480e-a845-75e9bac0b988 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:51.139231 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ff64c-8594-41c1-8d61-4f26a07904a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:51.140222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ff64c-8594-41c1-8d61-4f26a07904a2 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:51.140507 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ff64c-8594-41c1-8d61-4f26a07904a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:51.140780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ff64c-8594-41c1-8d61-4f26a07904a2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:51.154160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971ff64c-8594-41c1-8d61-4f26a07904a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:08:51.159727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-971ff64c-8594-41c1-8d61-4f26a07904a2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 00:08:51.159727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:51.159727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:51.159727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:51.159727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:51.159727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:51.159727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:51.159727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:51.159727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:51.159727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:51.159727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:51.159727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:51.159727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:08:51.159727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:08:51.159727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:08:51.159727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:08:51.160877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:08:51.160877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:08:51.160877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:08:51.160877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:08:51.160877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:51.160877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:51.160877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:51.160877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:51.160877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:51.160877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:51.160877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:51.160877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:51.160877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:51.160877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:51.160877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:51.160877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:51.161711 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:51.161711 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:51.161711 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:51.161711 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:51.161711 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:08:51.161711 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:08:51.161711 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:08:51.161711 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:08:51.161711 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:08:51.161711 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:08:51.161711 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 00:08:51.161711 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:51.161711 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 505/1029] 10.222.0.67 () {56 vars in 1120 bytes} [Wed Aug 31 00:08:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:51.163354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075f4ff-d761-4cf0-a3b4-849d0204161c None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:51.168133 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ff64c-8594-41c1-8d61-4f26a07904a2 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:51.207806 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51360146-e2d2-4bf7-aa1d-6e27ae525af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:51.208452 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51360146-e2d2-4bf7-aa1d-6e27ae525af2 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:51.208733 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51360146-e2d2-4bf7-aa1d-6e27ae525af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:51.209004 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51360146-e2d2-4bf7-aa1d-6e27ae525af2 None admin] PATH_INFO: `/v3/services/image` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:51.209599 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51360146-e2d2-4bf7-aa1d-6e27ae525af2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:51.211013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51360146-e2d2-4bf7-aa1d-6e27ae525af2 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:51.223927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:51.224102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-51360146-e2d2-4bf7-aa1d-6e27ae525af2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 00:08:51.224102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:51.224102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:08:51.224102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:08:51.224102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:08:51.224102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:08:51.224102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:08:51.224102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:51.224102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 00:08:51.224102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:51.224102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:08:51.224102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:51.224102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:51.224102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:51.224102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:51.224102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:51.224102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:51.225090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:51.225090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:51.225090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:51.225090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:51.225090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:51.225090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:51.225090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:08:51.225090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:08:51.225090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:08:51.225090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:08:51.225090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:51.225090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:51.225090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:51.225090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:51.225090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:51.225090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:51.228026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:51.228026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:51.228026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:51.228026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:51.228026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:51.228026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:51.228026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:51.228026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:51.228026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:51.228026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:51.228026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:08:51.228026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:51.228026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 00:08:51.228026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:51.228026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 525/1030] 10.222.0.67 () {56 vars in 1105 bytes} [Wed Aug 31 00:08:51 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:51.231921 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51360146-e2d2-4bf7-aa1d-6e27ae525af2 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:51.242362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:51.279241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ff64c-8594-41c1-8d61-4f26a07904a2 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:51.315094 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042d847-857c-4a91-949d-1d6d5a5dbac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:51.315704 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042d847-857c-4a91-949d-1d6d5a5dbac7 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:51.315991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042d847-857c-4a91-949d-1d6d5a5dbac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:51.316266 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042d847-857c-4a91-949d-1d6d5a5dbac7 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:51.316936 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042d847-857c-4a91-949d-1d6d5a5dbac7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:51.317243 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042d847-857c-4a91-949d-1d6d5a5dbac7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:51.318605 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042d847-857c-4a91-949d-1d6d5a5dbac7 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:51.327938 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 506/1031] 10.222.0.67 () {56 vars in 1115 bytes} [Wed Aug 31 00:08:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:51.333336 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042d847-857c-4a91-949d-1d6d5a5dbac7 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:51.345661 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51360146-e2d2-4bf7-aa1d-6e27ae525af2 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:51.380426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f26447-f507-4936-afea-965a9370bf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:51.381040 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f26447-f507-4936-afea-965a9370bf46 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:51.381318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f26447-f507-4936-afea-965a9370bf46 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:51.381591 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f26447-f507-4936-afea-965a9370bf46 None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:51.382218 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f26447-f507-4936-afea-965a9370bf46 None admin] RBAC: Adding query filter params (name=image) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:51.382525 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f26447-f507-4936-afea-965a9370bf46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:51.383914 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f26447-f507-4936-afea-965a9370bf46 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:51.390212 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 526/1032] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:51.395654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f26447-f507-4936-afea-965a9370bf46 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:51.447324 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042d847-857c-4a91-949d-1d6d5a5dbac7 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:51.461893 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:51.474604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:51.487675 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5274dc17-8fa1-4642-9c9b-58477ebd1420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:51.488303 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274dc17-8fa1-4642-9c9b-58477ebd1420 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:51.488583 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274dc17-8fa1-4642-9c9b-58477ebd1420 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:51.488933 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274dc17-8fa1-4642-9c9b-58477ebd1420 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:51.500292 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5274dc17-8fa1-4642-9c9b-58477ebd1420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:51.502001 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5274dc17-8fa1-4642-9c9b-58477ebd1420 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:51.510129 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f26447-f507-4936-afea-965a9370bf46 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:51.517046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-5274dc17-8fa1-4642-9c9b-58477ebd1420 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:51.517046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:51.517046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:51.517046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:51.517046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:51.517046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:51.517046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:51.517046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:51.517046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:51.517046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:51.517046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:51.517046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:51.517046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:51.517046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:51.517046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:51.517046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:51.518019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:51.518019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:51.518019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:51.518019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:51.518019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:51.518019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:51.518019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:51.518019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:51.518019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:51.518019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:51.518019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:51.518019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:51.518019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:51.518019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:51.518019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:51.518019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:51.518019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:51.519164 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:51.519164 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:51.519164 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:51.519164 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 507/1033] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:51.524147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5274dc17-8fa1-4642-9c9b-58477ebd1420 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:51.549157 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f5509-ac3a-4ef9-9a2b-89bb45e9e211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:51.549774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f5509-ac3a-4ef9-9a2b-89bb45e9e211 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:51.550062 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f5509-ac3a-4ef9-9a2b-89bb45e9e211 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:51.550333 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f5509-ac3a-4ef9-9a2b-89bb45e9e211 None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:51.550967 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f5509-ac3a-4ef9-9a2b-89bb45e9e211 None admin] RBAC: Adding query filter params (type=image) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:51.551303 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f5509-ac3a-4ef9-9a2b-89bb45e9e211 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:51.552672 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f5509-ac3a-4ef9-9a2b-89bb45e9e211 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:51.557917 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 527/1034] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:51 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:51.564019 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f5509-ac3a-4ef9-9a2b-89bb45e9e211 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:51.633028 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5274dc17-8fa1-4642-9c9b-58477ebd1420 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:51.671837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d134d-abb1-4b2f-8bcf-247b0d78ba1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:51.672462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d134d-abb1-4b2f-8bcf-247b0d78ba1a None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:51.672740 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d134d-abb1-4b2f-8bcf-247b0d78ba1a None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:51.672927 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f5509-ac3a-4ef9-9a2b-89bb45e9e211 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:51.673035 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d134d-abb1-4b2f-8bcf-247b0d78ba1a None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:51.673645 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d134d-abb1-4b2f-8bcf-247b0d78ba1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:51.673950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d134d-abb1-4b2f-8bcf-247b0d78ba1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:51.675359 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d134d-abb1-4b2f-8bcf-247b0d78ba1a None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:51.689016 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 508/1035] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:51.694145 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d134d-abb1-4b2f-8bcf-247b0d78ba1a None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:51.712984 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a47ead-0185-4feb-b577-35572da19a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:51.713598 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a47ead-0185-4feb-b577-35572da19a1e None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:51.713878 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a47ead-0185-4feb-b577-35572da19a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:51.714150 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a47ead-0185-4feb-b577-35572da19a1e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:51.714848 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a47ead-0185-4feb-b577-35572da19a1e None admin] RBAC: Adding query filter params (interface=public, service_id=1addca92a1d34446a3ccb3f30ff1308f, region_id=RegionOne) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:51.715190 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a47ead-0185-4feb-b577-35572da19a1e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:51.716790 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a47ead-0185-4feb-b577-35572da19a1e None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:51.718409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:51.721381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:51.723953 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 528/1036] 10.222.0.67 () {56 vars in 1251 bytes} [Wed Aug 31 00:08:51 2022] GET /identity/v3/endpoints?service_id=1addca92a1d34446a3ccb3f30ff1308f&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:51.728965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a47ead-0185-4feb-b577-35572da19a1e None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:51.807764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d134d-abb1-4b2f-8bcf-247b0d78ba1a None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:51.838266 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a47ead-0185-4feb-b577-35572da19a1e None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:51.853190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc6cdc-4141-4420-af5a-b8ca2691d3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:51.853819 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc6cdc-4141-4420-af5a-b8ca2691d3d0 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:51.854100 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc6cdc-4141-4420-af5a-b8ca2691d3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:51.854622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc6cdc-4141-4420-af5a-b8ca2691d3d0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:51.862493 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fc6cdc-4141-4420-af5a-b8ca2691d3d0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:51.864098 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fc6cdc-4141-4420-af5a-b8ca2691d3d0 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:51.876298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-d8fc6cdc-4141-4420-af5a-b8ca2691d3d0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:08:51.876298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:51.876298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:51.876298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:51.876298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:51.876298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:51.876298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:51.876298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:51.876298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:51.876298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:51.876298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:51.876298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:51.876298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:08:51.876298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:08:51.876298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:08:51.876298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:08:51.877549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:51.877549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:51.877549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:08:51.877549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:51.877549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:08:51.877549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:51.877549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:51.877549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:51.877549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:51.877549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:51.877549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:51.877549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:51.877549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:51.877549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:51.877549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:51.877549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:51.877549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:51.877549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:51.878520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:51.878520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:51.878520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:08:51.878520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:08:51.878520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:08:51.878520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:08:51.878520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:08:51.878520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:08:51.878520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:08:51.878520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:51.879081 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c723e-0d27-44ce-b00c-97a7c3962e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:51.879081 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c723e-0d27-44ce-b00c-97a7c3962e06 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:51.879081 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c723e-0d27-44ce-b00c-97a7c3962e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:51.879081 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c723e-0d27-44ce-b00c-97a7c3962e06 None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:51.879386 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 509/1037] 10.222.0.67 () {56 vars in 1093 bytes} [Wed Aug 31 00:08:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:51.879888 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c723e-0d27-44ce-b00c-97a7c3962e06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:51.881083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c723e-0d27-44ce-b00c-97a7c3962e06 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:51.883686 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc6cdc-4141-4420-af5a-b8ca2691d3d0 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:51.895073 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 529/1038] 10.222.0.67 () {56 vars in 1087 bytes} [Wed Aug 31 00:08:51 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:08:51.997033 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:52.016650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc6cdc-4141-4420-af5a-b8ca2691d3d0 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:52.053043 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bdcbcf-d919-4576-b764-17b8399ffaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:52.053652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdcbcf-d919-4576-b764-17b8399ffaff None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:52.053930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdcbcf-d919-4576-b764-17b8399ffaff None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:52.054197 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdcbcf-d919-4576-b764-17b8399ffaff None admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:52.054929 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bdcbcf-d919-4576-b764-17b8399ffaff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:52.055262 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bdcbcf-d919-4576-b764-17b8399ffaff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:52.056685 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bdcbcf-d919-4576-b764-17b8399ffaff None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:52.075121 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 510/1039] 10.222.0.67 () {56 vars in 1133 bytes} [Wed Aug 31 00:08:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:52.081778 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c723e-0d27-44ce-b00c-97a7c3962e06 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:52.146051 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:52.187996 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c723e-0d27-44ce-b00c-97a7c3962e06 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:52.226674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfee15-3881-4b1a-8a37-48a91e601401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:52.227321 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfee15-3881-4b1a-8a37-48a91e601401 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:52.227604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfee15-3881-4b1a-8a37-48a91e601401 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:52.227872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfee15-3881-4b1a-8a37-48a91e601401 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:52.302985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdfee15-3881-4b1a-8a37-48a91e601401 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:52.304544 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdfee15-3881-4b1a-8a37-48a91e601401 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:52.367095 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:52.415610 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 530/1040] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:08:52 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:08:52.416917 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bdcbcf-d919-4576-b764-17b8399ffaff None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:52.518827 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bdcbcf-d919-4576-b764-17b8399ffaff None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:52.555486 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a873cf-e571-4346-95b0-b672f2220c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:52.556120 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a873cf-e571-4346-95b0-b672f2220c44 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:52.556395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a873cf-e571-4346-95b0-b672f2220c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:52.556664 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a873cf-e571-4346-95b0-b672f2220c44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:52.567073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a873cf-e571-4346-95b0-b672f2220c44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:52.568512 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a873cf-e571-4346-95b0-b672f2220c44 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:52.570273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:52.584916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-97a873cf-e571-4346-95b0-b672f2220c44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:52.584916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:52.584916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:52.584916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:52.584916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:52.584916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:52.584916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:52.584916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:52.584916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:52.584916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:52.584916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:52.584916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:52.584916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:52.584916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:52.584916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:52.584916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:52.586087 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:52.586087 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:52.586087 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:52.586087 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:52.586087 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:52.586087 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:52.586087 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:52.586087 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:52.586087 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:52.586087 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:52.586087 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:52.586087 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:52.586087 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:52.586087 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:52.586087 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:52.586087 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:52.586087 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:52.587192 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:52.587192 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:52.587192 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:52.593546 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 511/1041] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:52.594815 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfee15-3881-4b1a-8a37-48a91e601401 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:52.634019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f2c12-b7a3-4d05-a8ca-fd167e13ca66 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:52.634214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f2c12-b7a3-4d05-a8ca-fd167e13ca66 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:52.634486 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f2c12-b7a3-4d05-a8ca-fd167e13ca66 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:52.635996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 512/1042] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:08:52.641280 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66352bc3-0150-4b05-bb25-9a7e309174a5 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:52.641559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66352bc3-0150-4b05-bb25-9a7e309174a5 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:52.641825 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66352bc3-0150-4b05-bb25-9a7e309174a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:52.660252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-66352bc3-0150-4b05-bb25-9a7e309174a5 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:52.661429 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-66352bc3-0150-4b05-bb25-9a7e309174a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:52.700164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfee15-3881-4b1a-8a37-48a91e601401 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:52.733485 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 513/1043] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:08:52.735361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3dbc5-e537-45cd-83bc-bda957082054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:52.735980 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3dbc5-e537-45cd-83bc-bda957082054 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:52.736337 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3dbc5-e537-45cd-83bc-bda957082054 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:52.736613 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3dbc5-e537-45cd-83bc-bda957082054 None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:52.737259 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3dbc5-e537-45cd-83bc-bda957082054 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:52.737565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3dbc5-e537-45cd-83bc-bda957082054 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:52.738927 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3dbc5-e537-45cd-83bc-bda957082054 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:52.755077 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 531/1044] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:52.757016 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66352bc3-0150-4b05-bb25-9a7e309174a5 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:52.863444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66352bc3-0150-4b05-bb25-9a7e309174a5 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:52.889342 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:52.902413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4375a6-a1b6-46e8-a8fc-aeac36971118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:52.903061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4375a6-a1b6-46e8-a8fc-aeac36971118 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:52.903360 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4375a6-a1b6-46e8-a8fc-aeac36971118 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:52.903704 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4375a6-a1b6-46e8-a8fc-aeac36971118 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:52.911256 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4375a6-a1b6-46e8-a8fc-aeac36971118 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:52.912682 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4375a6-a1b6-46e8-a8fc-aeac36971118 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:52.923975 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-cc4375a6-a1b6-46e8-a8fc-aeac36971118 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:52.923975 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:52.923975 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:52.923975 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:52.923975 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:52.923975 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:52.923975 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:52.923975 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:52.923975 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:52.923975 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:52.923975 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:52.923975 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:52.923975 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:08:52.923975 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:08:52.923975 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:08:52.924859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:08:52.924859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:52.924859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:52.924859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:52.924859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:52.924859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:52.924859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:52.924859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:52.924859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:52.924859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:52.924859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:52.924859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:52.924859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:52.924859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:52.924859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:52.924859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:52.924859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:52.925835 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:08:52.925835 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:08:52.925835 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:08:52.925835 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:08:52.925835 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:08:52.925835 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:08:52.925835 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:52.925835 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:52.927391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 514/1045] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:52.931314 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3dbc5-e537-45cd-83bc-bda957082054 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:52.958306 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:53.013367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4375a6-a1b6-46e8-a8fc-aeac36971118 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:53.054908 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3dbc5-e537-45cd-83bc-bda957082054 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:53.092927 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf7923-66fc-4bf6-ac18-2b00f477d399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:53.093549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf7923-66fc-4bf6-ac18-2b00f477d399 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:53.093802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf7923-66fc-4bf6-ac18-2b00f477d399 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:53.094073 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf7923-66fc-4bf6-ac18-2b00f477d399 None admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:53.094759 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf7923-66fc-4bf6-ac18-2b00f477d399 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:53.095083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf7923-66fc-4bf6-ac18-2b00f477d399 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:53.096515 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf7923-66fc-4bf6-ac18-2b00f477d399 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:53.109358 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 532/1046] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:08:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:53.115245 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf7923-66fc-4bf6-ac18-2b00f477d399 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:53.121997 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4375a6-a1b6-46e8-a8fc-aeac36971118 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:53.157068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44133c-2406-4436-b6af-a786aeb9f35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:53.157637 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44133c-2406-4436-b6af-a786aeb9f35d None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:53.157916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44133c-2406-4436-b6af-a786aeb9f35d None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:53.158183 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44133c-2406-4436-b6af-a786aeb9f35d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:53.170046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44133c-2406-4436-b6af-a786aeb9f35d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:08:53.174729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-9f44133c-2406-4436-b6af-a786aeb9f35d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:08:53.174729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:53.174729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:53.174729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:53.174729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:53.174729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:53.174729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:53.174729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:53.174729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:53.174729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:53.174729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:53.174729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:53.174729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:08:53.174729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:08:53.174729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:08:53.174729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:08:53.175863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:08:53.175863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:08:53.175863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:08:53.175863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:08:53.175863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:53.175863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:53.175863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:53.175863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:53.175863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:53.175863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:53.175863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:53.175863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:53.175863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:53.175863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:53.175863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:53.175863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:53.177351 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:53.177351 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:53.177351 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:53.177351 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:53.177351 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:08:53.177351 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:08:53.177351 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:08:53.177351 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:08:53.177351 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:08:53.177351 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:08:53.177351 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:08:53.177351 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:53.177351 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 515/1047] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:08:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:53.181620 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44133c-2406-4436-b6af-a786aeb9f35d None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:53.203390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:53.223062 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf7923-66fc-4bf6-ac18-2b00f477d399 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:53.229917 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:53.260056 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c1b13-1866-45b2-a917-f7fb2bc0683e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:53.260856 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c1b13-1866-45b2-a917-f7fb2bc0683e None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:53.261205 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c1b13-1866-45b2-a917-f7fb2bc0683e None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:53.261969 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c1b13-1866-45b2-a917-f7fb2bc0683e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:53.262993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1c1b13-1866-45b2-a917-f7fb2bc0683e None admin] RBAC: Adding query filter params (role.id=39f496940d6844c8a43185fbd5fc1d21, scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb, user.id=3aa53ae5a30146f69d7d6eefe4290dba) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:53.264118 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1c1b13-1866-45b2-a917-f7fb2bc0683e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:53.266425 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1c1b13-1866-45b2-a917-f7fb2bc0683e None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:53.272426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 533/1048] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:08:53 2022] GET /identity/v3/role_assignments?user.id=3aa53ae5a30146f69d7d6eefe4290dba&scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb&role.id=39f496940d6844c8a43185fbd5fc1d21 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:53.293748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44133c-2406-4436-b6af-a786aeb9f35d None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:53.333935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad462e90-b0a7-43d6-aac6-734a05d60b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:53.334651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad462e90-b0a7-43d6-aac6-734a05d60b9c None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:53.334999 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad462e90-b0a7-43d6-aac6-734a05d60b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:53.335395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad462e90-b0a7-43d6-aac6-734a05d60b9c None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:53.336146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad462e90-b0a7-43d6-aac6-734a05d60b9c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:53.336518 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad462e90-b0a7-43d6-aac6-734a05d60b9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:53.337971 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad462e90-b0a7-43d6-aac6-734a05d60b9c None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:53.348593 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 516/1049] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:08:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:53.354331 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c1b13-1866-45b2-a917-f7fb2bc0683e None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:53.464740 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c1b13-1866-45b2-a917-f7fb2bc0683e None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:53.498951 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac6941-b474-4cd2-bf9a-56e8af6ba465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:53.499615 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac6941-b474-4cd2-bf9a-56e8af6ba465 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:53.499891 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac6941-b474-4cd2-bf9a-56e8af6ba465 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:53.500158 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac6941-b474-4cd2-bf9a-56e8af6ba465 None admin] PATH_INFO: `/v3/users/3fef4ee8dfd34241a8551e2bbcca7048` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:53.513987 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac6941-b474-4cd2-bf9a-56e8af6ba465 None admin] RBAC: Authorizing `identity:get_user(user_id=3fef4ee8dfd34241a8551e2bbcca7048)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:53.515453 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac6941-b474-4cd2-bf9a-56e8af6ba465 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:53.517327 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 534/1050] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:08:53 2022] GET /identity/v3/users/3fef4ee8dfd34241a8551e2bbcca7048 => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:53.522916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad462e90-b0a7-43d6-aac6-734a05d60b9c None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:53.632639 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad462e90-b0a7-43d6-aac6-734a05d60b9c None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:53.640928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:53.669875 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75420ce-81b6-4d03-b81f-143414962bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:53.670581 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75420ce-81b6-4d03-b81f-143414962bfd None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:53.670919 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75420ce-81b6-4d03-b81f-143414962bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:53.671279 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75420ce-81b6-4d03-b81f-143414962bfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:53.741237 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75420ce-81b6-4d03-b81f-143414962bfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:53.742680 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75420ce-81b6-4d03-b81f-143414962bfd None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:53.847428 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 517/1051] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:08:53 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:08:53.849097 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac6941-b474-4cd2-bf9a-56e8af6ba465 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:53.852971 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:53.894783 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771dbcd-4751-42b2-bec0-2ef849f4235c None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:53.895155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771dbcd-4751-42b2-bec0-2ef849f4235c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:53.895359 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771dbcd-4751-42b2-bec0-2ef849f4235c None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:53.896867 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 518/1052] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:08:53.901629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a50d8-ae29-4b2e-a96f-b2626dbf1950 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:53.901919 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a50d8-ae29-4b2e-a96f-b2626dbf1950 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:53.902198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a50d8-ae29-4b2e-a96f-b2626dbf1950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:53.917096 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:53.924436 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-ab1a50d8-ae29-4b2e-a96f-b2626dbf1950 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:53.925814 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-ab1a50d8-ae29-4b2e-a96f-b2626dbf1950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:53.965689 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac6941-b474-4cd2-bf9a-56e8af6ba465 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:53.999579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 519/1053] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:08:54.012434 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29b2be-46b1-4f27-a8fd-0eb6b4e6f529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:54.013936 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29b2be-46b1-4f27-a8fd-0eb6b4e6f529 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:54.014373 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29b2be-46b1-4f27-a8fd-0eb6b4e6f529 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:54.014794 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29b2be-46b1-4f27-a8fd-0eb6b4e6f529 None admin] PATH_INFO: `/v3/projects/0cd36903cbe141c6abc9a3df855737c6` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:54.016135 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d29b2be-46b1-4f27-a8fd-0eb6b4e6f529 None admin] RBAC: Authorizing `identity:get_project(project_id=0cd36903cbe141c6abc9a3df855737c6)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:54.017716 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d29b2be-46b1-4f27-a8fd-0eb6b4e6f529 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:54.024341 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 535/1054] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:08:53 2022] GET /identity/v3/projects/0cd36903cbe141c6abc9a3df855737c6 => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:54.026030 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a50d8-ae29-4b2e-a96f-b2626dbf1950 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:54.130158 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a50d8-ae29-4b2e-a96f-b2626dbf1950 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:54.165497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a992a6-513f-44e7-b365-7317f0ca7179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:54.166094 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a992a6-513f-44e7-b365-7317f0ca7179 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:54.166368 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a992a6-513f-44e7-b365-7317f0ca7179 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:54.166635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a992a6-513f-44e7-b365-7317f0ca7179 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:54.167397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a992a6-513f-44e7-b365-7317f0ca7179 None admin] RBAC: Adding query filter params (role.id=34df06ce84ad4cf8b3bd1a4490228017, scope.project.id=0cd36903cbe141c6abc9a3df855737c6, user.id=3fef4ee8dfd34241a8551e2bbcca7048) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:54.167702 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a992a6-513f-44e7-b365-7317f0ca7179 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:54.169108 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a992a6-513f-44e7-b365-7317f0ca7179 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:54.173106 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:54.183505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 520/1055] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:08:54 2022] GET /identity/v3/role_assignments?user.id=3fef4ee8dfd34241a8551e2bbcca7048&scope.project.id=0cd36903cbe141c6abc9a3df855737c6&role.id=34df06ce84ad4cf8b3bd1a4490228017 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:54.278360 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29b2be-46b1-4f27-a8fd-0eb6b4e6f529 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:54.368835 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:54.401309 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29b2be-46b1-4f27-a8fd-0eb6b4e6f529 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:54.436007 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da17a6f-eea5-451a-a07a-4b6957ec47d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:54.436639 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da17a6f-eea5-451a-a07a-4b6957ec47d3 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:54.436921 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da17a6f-eea5-451a-a07a-4b6957ec47d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:54.437193 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da17a6f-eea5-451a-a07a-4b6957ec47d3 None admin] PATH_INFO: `/v3/services/image` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:54.437774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da17a6f-eea5-451a-a07a-4b6957ec47d3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:54.439163 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da17a6f-eea5-451a-a07a-4b6957ec47d3 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:54.447964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-4da17a6f-eea5-451a-a07a-4b6957ec47d3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 00:08:54.447964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:54.447964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:08:54.447964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:08:54.447964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:08:54.447964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:08:54.447964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:08:54.447964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:54.447964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 00:08:54.447964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:54.447964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:08:54.447964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:54.447964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:54.447964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:54.447964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:54.447964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:54.447964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:54.448946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:54.448946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:54.448946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:54.448946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:54.448946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:54.448946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:54.448946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:08:54.448946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:08:54.448946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:08:54.448946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:08:54.448946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:54.448946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:54.448946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:54.448946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:54.448946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:54.448946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:54.449788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:54.449788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:54.449788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:54.449788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:54.449788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:54.449788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:54.449788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:54.449788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:54.449788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:54.449788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:54.449788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:08:54.449788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:08:54.449788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 00:08:54.449788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:54.449788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 536/1056] 10.222.0.67 () {56 vars in 1105 bytes} [Wed Aug 31 00:08:54 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:54.456915 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a992a6-513f-44e7-b365-7317f0ca7179 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:54.558366 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a992a6-513f-44e7-b365-7317f0ca7179 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:54.595117 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e7ef0-72ff-4589-aedc-28f23ecb8acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:54.595879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e7ef0-72ff-4589-aedc-28f23ecb8acd None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:54.596279 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e7ef0-72ff-4589-aedc-28f23ecb8acd None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:54.596653 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e7ef0-72ff-4589-aedc-28f23ecb8acd None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:54.597389 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e7ef0-72ff-4589-aedc-28f23ecb8acd None admin] RBAC: Adding query filter params (name=image) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:54.597791 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e7ef0-72ff-4589-aedc-28f23ecb8acd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:54.599274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e7ef0-72ff-4589-aedc-28f23ecb8acd None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:54.609652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 521/1057] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:54.610872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da17a6f-eea5-451a-a07a-4b6957ec47d3 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:54.710789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da17a6f-eea5-451a-a07a-4b6957ec47d3 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:54.744558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c8f21-81ef-44a4-9b04-b64dc66c964d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:54.745166 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c8f21-81ef-44a4-9b04-b64dc66c964d None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:54.745438 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c8f21-81ef-44a4-9b04-b64dc66c964d None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:54.745702 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c8f21-81ef-44a4-9b04-b64dc66c964d None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:54.746324 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c8f21-81ef-44a4-9b04-b64dc66c964d None admin] RBAC: Adding query filter params (type=image) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:54.746622 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c8f21-81ef-44a4-9b04-b64dc66c964d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:54.748772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c8f21-81ef-44a4-9b04-b64dc66c964d None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:54.750805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:54.755132 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 537/1058] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:54.761218 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e7ef0-72ff-4589-aedc-28f23ecb8acd None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:54.825831 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:54.866916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e7ef0-72ff-4589-aedc-28f23ecb8acd None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:54.902822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4856c8e4-7d8f-49b9-8c85-089ae1154666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:54.903600 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856c8e4-7d8f-49b9-8c85-089ae1154666 None admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:54.904571 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856c8e4-7d8f-49b9-8c85-089ae1154666 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:54.904939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856c8e4-7d8f-49b9-8c85-089ae1154666 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:54.905708 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4856c8e4-7d8f-49b9-8c85-089ae1154666 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:54.907191 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4856c8e4-7d8f-49b9-8c85-089ae1154666 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:54.930806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 522/1059] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:08:54 2022] POST /identity/v3/endpoints => generated 327 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:08:55.186432 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b97e65-7114-450a-a8c8-f1c4bfa45cca None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:55.186827 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b97e65-7114-450a-a8c8-f1c4bfa45cca None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:55.187014 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b97e65-7114-450a-a8c8-f1c4bfa45cca None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:55.188529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 538/1060] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:55.193574 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de342ecf-8c91-4d8b-a13e-db8046d6bae5 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:55.194043 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de342ecf-8c91-4d8b-a13e-db8046d6bae5 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:55.194137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de342ecf-8c91-4d8b-a13e-db8046d6bae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:55.212982 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-de342ecf-8c91-4d8b-a13e-db8046d6bae5 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:55.214156 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-de342ecf-8c91-4d8b-a13e-db8046d6bae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:55.293312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 523/1061] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:55.564291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b97e65-7114-450a-a8c8-f1c4bfa45cca None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:55.665258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b97e65-7114-450a-a8c8-f1c4bfa45cca None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:55.697794 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c445fc-6f42-40c4-8655-8faf9212f48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:55.698422 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c445fc-6f42-40c4-8655-8faf9212f48a None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:55.698697 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c445fc-6f42-40c4-8655-8faf9212f48a None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:55.698962 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c445fc-6f42-40c4-8655-8faf9212f48a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:55.709743 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c445fc-6f42-40c4-8655-8faf9212f48a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:08:55.714409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-47c445fc-6f42-40c4-8655-8faf9212f48a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 00:08:55.714409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:55.714409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:55.714409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:55.714409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:55.714409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:55.714409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:55.714409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:55.714409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:55.714409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:55.714409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:55.714409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:55.714409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:08:55.714409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:08:55.714409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:08:55.714409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:08:55.715561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:08:55.715561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:08:55.715561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:08:55.715561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:08:55.715561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:55.715561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:55.715561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:55.715561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:55.715561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:55.715561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:55.715561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:55.715561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:55.715561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:55.715561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:55.715561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:55.715561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:55.717040 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:55.717040 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:55.717040 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:55.717040 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:55.717040 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:08:55.717040 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:08:55.717040 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:08:55.717040 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:08:55.717040 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:08:55.717040 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:08:55.717040 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 00:08:55.717040 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:55.719117 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 539/1062] 10.222.0.67 () {56 vars in 1120 bytes} [Wed Aug 31 00:08:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:55.722545 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de342ecf-8c91-4d8b-a13e-db8046d6bae5 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:55.769245 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:55.825311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de342ecf-8c91-4d8b-a13e-db8046d6bae5 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:55.859239 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ea6dd-7922-454f-a23f-5fba436c5edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:55.859846 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ea6dd-7922-454f-a23f-5fba436c5edd None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:55.860122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ea6dd-7922-454f-a23f-5fba436c5edd None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:55.860390 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ea6dd-7922-454f-a23f-5fba436c5edd None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:55.861048 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ea6dd-7922-454f-a23f-5fba436c5edd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:55.861347 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ea6dd-7922-454f-a23f-5fba436c5edd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:55.862703 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ea6dd-7922-454f-a23f-5fba436c5edd None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:55.870815 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 524/1063] 10.222.0.67 () {56 vars in 1115 bytes} [Wed Aug 31 00:08:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:55.877582 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c445fc-6f42-40c4-8655-8faf9212f48a None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:55.938175 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:55.985126 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c445fc-6f42-40c4-8655-8faf9212f48a None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:56.020405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee07399-feeb-4f86-a281-1822966d891e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:56.021034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee07399-feeb-4f86-a281-1822966d891e None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:56.021314 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee07399-feeb-4f86-a281-1822966d891e None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:56.021580 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee07399-feeb-4f86-a281-1822966d891e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:56.033551 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee07399-feeb-4f86-a281-1822966d891e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:56.035065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee07399-feeb-4f86-a281-1822966d891e None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:56.051079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-2ee07399-feeb-4f86-a281-1822966d891e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:56.051079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:56.051079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:56.051079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:56.051079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:56.051079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:56.051079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:56.051079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:56.051079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:56.051079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:56.051079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:56.051079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:56.051079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:56.051079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:56.051079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:56.051079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:56.052089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:56.052089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:56.052089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:56.052089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:56.052089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:56.052089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:56.052089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:56.052089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:56.052089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:56.052089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:56.052089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:56.052089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:56.052089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:56.052089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:56.052089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:56.052089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:56.052089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:56.052998 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:56.052998 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:56.052998 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:56.056966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 540/1064] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:56.058370 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ea6dd-7922-454f-a23f-5fba436c5edd None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:56.122845 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4cc61-b3e5-4362-a430-d918c424e377 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:56.123357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4cc61-b3e5-4362-a430-d918c424e377 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:56.123467 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4cc61-b3e5-4362-a430-d918c424e377 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:56.124777 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 541/1065] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:08:56.130054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82655fcc-f63e-4128-880f-9b25e0e959e4 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:56.130344 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82655fcc-f63e-4128-880f-9b25e0e959e4 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:56.130623 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82655fcc-f63e-4128-880f-9b25e0e959e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:56.148615 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-82655fcc-f63e-4128-880f-9b25e0e959e4 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:56.149885 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-82655fcc-f63e-4128-880f-9b25e0e959e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:56.163533 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ea6dd-7922-454f-a23f-5fba436c5edd None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:56.173387 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:56.201192 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f37432-5ebd-4356-9ebd-d5983b9f52d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:56.201810 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f37432-5ebd-4356-9ebd-d5983b9f52d1 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:56.202058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f37432-5ebd-4356-9ebd-d5983b9f52d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:56.202311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f37432-5ebd-4356-9ebd-d5983b9f52d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:56.202948 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f37432-5ebd-4356-9ebd-d5983b9f52d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:56.203280 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f37432-5ebd-4356-9ebd-d5983b9f52d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:56.204634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f37432-5ebd-4356-9ebd-d5983b9f52d1 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:56.217629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 525/1066] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:56.223456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f37432-5ebd-4356-9ebd-d5983b9f52d1 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:56.226119 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 542/1067] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:08:56.327460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f37432-5ebd-4356-9ebd-d5983b9f52d1 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:56.361959 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c00e2d-b954-479a-bd9c-f2658bdb5b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:56.362580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c00e2d-b954-479a-bd9c-f2658bdb5b1c None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:56.362852 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c00e2d-b954-479a-bd9c-f2658bdb5b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:56.363142 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c00e2d-b954-479a-bd9c-f2658bdb5b1c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:56.370108 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c00e2d-b954-479a-bd9c-f2658bdb5b1c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:56.371600 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c00e2d-b954-479a-bd9c-f2658bdb5b1c None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:56.383665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-12c00e2d-b954-479a-bd9c-f2658bdb5b1c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:08:56.383665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:56.383665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:56.383665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:56.383665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:56.383665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:56.383665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:56.383665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:56.383665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:56.383665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:56.383665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:56.383665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:56.383665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:08:56.383665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:08:56.383665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:08:56.383665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:08:56.384604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:56.384604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:56.384604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:08:56.384604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:56.384604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:08:56.384604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:08:56.384604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:56.384604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:56.384604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:56.384604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:56.384604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:56.384604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:56.384604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:56.384604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:56.384604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:56.384604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:56.384604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:56.384604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:56.385556 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:56.385556 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:56.385556 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:08:56.385556 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:08:56.385556 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:08:56.385556 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:08:56.385556 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:08:56.385556 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:08:56.385556 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:08:56.385556 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:56.385556 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 526/1068] 10.222.0.67 () {56 vars in 1093 bytes} [Wed Aug 31 00:08:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:56.391264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82655fcc-f63e-4128-880f-9b25e0e959e4 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:56.441999 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:56.500931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82655fcc-f63e-4128-880f-9b25e0e959e4 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:56.505432 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c00e2d-b954-479a-bd9c-f2658bdb5b1c None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:56.535262 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:56.535448 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b7109-8be3-4933-ad42-2033466faabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:56.535914 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b7109-8be3-4933-ad42-2033466faabe None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:56.536212 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b7109-8be3-4933-ad42-2033466faabe None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:56.536492 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b7109-8be3-4933-ad42-2033466faabe None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:56.537183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b7109-8be3-4933-ad42-2033466faabe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:56.537539 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b7109-8be3-4933-ad42-2033466faabe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:56.539004 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b7109-8be3-4933-ad42-2033466faabe None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:56.565156 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 543/1069] 10.222.0.67 () {56 vars in 1133 bytes} [Wed Aug 31 00:08:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:56.567400 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b7109-8be3-4933-ad42-2033466faabe None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:56.619812 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c00e2d-b954-479a-bd9c-f2658bdb5b1c None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:56.663609 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9adbf-d176-4467-8a21-22fae5eafd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:56.664567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9adbf-d176-4467-8a21-22fae5eafd9b None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:56.664850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9adbf-d176-4467-8a21-22fae5eafd9b None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:56.665121 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9adbf-d176-4467-8a21-22fae5eafd9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:56.675138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:56.676715 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e9adbf-d176-4467-8a21-22fae5eafd9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:08:56.678681 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b7109-8be3-4933-ad42-2033466faabe None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:56.682155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-96e9adbf-d176-4467-8a21-22fae5eafd9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:08:56.682155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:56.682155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:56.682155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:56.682155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:56.682155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:56.682155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:56.682155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:56.682155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:56.682155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:56.682155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:56.682155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:56.682155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:08:56.682155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:08:56.682155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:08:56.682155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:08:56.683355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:08:56.683355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:08:56.683355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:08:56.683355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:08:56.683355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:56.683355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:56.683355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:56.683355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:56.683355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:56.683355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:56.683355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:56.683355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:56.683355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:56.683355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:56.683355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:56.683355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:56.684214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:56.684214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:56.684214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:56.684214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:56.684214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:08:56.684214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:08:56.684214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:08:56.684214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:08:56.684214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:08:56.684214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:08:56.684214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:08:56.684214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:56.688855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 527/1070] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:08:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:56.689484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9adbf-d176-4467-8a21-22fae5eafd9b None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:56.718172 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb167ab-7a4c-4063-8929-422a80d14181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:56.718795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb167ab-7a4c-4063-8929-422a80d14181 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:56.719107 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb167ab-7a4c-4063-8929-422a80d14181 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:56.719643 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb167ab-7a4c-4063-8929-422a80d14181 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:56.730952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb167ab-7a4c-4063-8929-422a80d14181 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:56.733124 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb167ab-7a4c-4063-8929-422a80d14181 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:56.749894 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-6fb167ab-7a4c-4063-8929-422a80d14181 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:56.749894 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:56.749894 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:56.749894 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:56.749894 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:56.749894 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:56.749894 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:56.749894 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:56.749894 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:56.749894 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:56.749894 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:56.749894 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:56.749894 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:56.749894 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:56.749894 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:56.749894 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:56.750899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:56.750899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:56.750899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:56.750899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:56.750899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:56.750899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:56.750899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:56.750899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:56.750899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:56.750899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:56.750899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:56.750899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:56.750899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:56.750899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:56.750899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:56.750899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:56.750899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:56.752125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:56.752125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:56.752125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:56.756026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 544/1071] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:56.757258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb167ab-7a4c-4063-8929-422a80d14181 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:56.785658 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:56.799678 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9adbf-d176-4467-8a21-22fae5eafd9b None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:56.835785 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8795b-f2d8-42ae-8541-0abaa5a77a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:56.836533 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8795b-f2d8-42ae-8541-0abaa5a77a43 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:56.836907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8795b-f2d8-42ae-8541-0abaa5a77a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:56.837543 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8795b-f2d8-42ae-8541-0abaa5a77a43 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:56.838058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8795b-f2d8-42ae-8541-0abaa5a77a43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:56.838453 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8795b-f2d8-42ae-8541-0abaa5a77a43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:56.839977 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8795b-f2d8-42ae-8541-0abaa5a77a43 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:56.849312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 528/1072] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:08:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:56.858113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8795b-f2d8-42ae-8541-0abaa5a77a43 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:56.870115 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb167ab-7a4c-4063-8929-422a80d14181 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:56.909385 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af65a8-0e54-4daa-9bba-1aceb206b02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:56.910046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af65a8-0e54-4daa-9bba-1aceb206b02c None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:56.910315 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af65a8-0e54-4daa-9bba-1aceb206b02c None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:56.910591 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af65a8-0e54-4daa-9bba-1aceb206b02c None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:56.911268 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af65a8-0e54-4daa-9bba-1aceb206b02c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:56.911578 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af65a8-0e54-4daa-9bba-1aceb206b02c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:56.913026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af65a8-0e54-4daa-9bba-1aceb206b02c None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:56.925990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 545/1073] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:56.928392 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d90c6c-5796-4d6d-a5ab-01e5e010532d None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:56.928679 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d90c6c-5796-4d6d-a5ab-01e5e010532d None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:56.929089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d90c6c-5796-4d6d-a5ab-01e5e010532d None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:56.930831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 546/1074] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:56.931838 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d90c6c-5796-4d6d-a5ab-01e5e010532d None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:56.947518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:56.965146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8795b-f2d8-42ae-8541-0abaa5a77a43 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:57.004868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c773ae4-ab14-4063-b405-5c8b4900e51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:57.005504 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c773ae4-ab14-4063-b405-5c8b4900e51f None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:57.005792 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c773ae4-ab14-4063-b405-5c8b4900e51f None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:57.006067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c773ae4-ab14-4063-b405-5c8b4900e51f None admin] PATH_INFO: `/v3/users/3fef4ee8dfd34241a8551e2bbcca7048` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:57.007236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c773ae4-ab14-4063-b405-5c8b4900e51f None admin] RBAC: Authorizing `identity:get_user(user_id=3fef4ee8dfd34241a8551e2bbcca7048)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:57.008662 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c773ae4-ab14-4063-b405-5c8b4900e51f None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:57.010363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 529/1075] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:08:56 2022] GET /identity/v3/users/3fef4ee8dfd34241a8551e2bbcca7048 => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:57.011580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867fb21-fcaf-48d1-a6c2-f48f194cce08 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:57.011874 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867fb21-fcaf-48d1-a6c2-f48f194cce08 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:57.012148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867fb21-fcaf-48d1-a6c2-f48f194cce08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:57.030796 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-2867fb21-fcaf-48d1-a6c2-f48f194cce08 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:57.032017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-2867fb21-fcaf-48d1-a6c2-f48f194cce08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:57.042527 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d90c6c-5796-4d6d-a5ab-01e5e010532d None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:57.045206 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:57.077323 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e6cad-39ab-4504-9dd0-224c5271e72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:57.077954 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e6cad-39ab-4504-9dd0-224c5271e72f None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:57.078236 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e6cad-39ab-4504-9dd0-224c5271e72f None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:57.078510 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e6cad-39ab-4504-9dd0-224c5271e72f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:57.085148 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e6cad-39ab-4504-9dd0-224c5271e72f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:57.086584 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e6cad-39ab-4504-9dd0-224c5271e72f None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:57.097492 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-163e6cad-39ab-4504-9dd0-224c5271e72f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:57.097492 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:57.097492 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:57.097492 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:57.097492 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:57.097492 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:57.097492 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:57.097492 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:57.097492 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:57.097492 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:57.097492 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:57.097492 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:57.097492 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:08:57.097492 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:08:57.097492 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:08:57.098504 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:08:57.098504 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:57.098504 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:57.098504 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:57.098504 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:57.098504 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:57.098504 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:57.098504 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:57.098504 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:57.098504 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:57.098504 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:57.098504 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:57.098504 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:57.098504 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:57.098504 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:57.098504 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:57.098504 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:57.099517 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:08:57.099517 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:08:57.099517 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:08:57.099517 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:08:57.099517 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:08:57.099517 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:08:57.099517 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:08:57.099517 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:57.099517 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 547/1076] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:08:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:57.100010 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e6cad-39ab-4504-9dd0-224c5271e72f None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:57.101567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 530/1077] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:57.104853 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867fb21-fcaf-48d1-a6c2-f48f194cce08 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:57.174318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:57.206154 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e6cad-39ab-4504-9dd0-224c5271e72f None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:57.214876 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867fb21-fcaf-48d1-a6c2-f48f194cce08 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:57.242302 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110e13c-052f-4978-9749-6052c52fdb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:57.242931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110e13c-052f-4978-9749-6052c52fdb9f None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:57.243235 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110e13c-052f-4978-9749-6052c52fdb9f None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:57.243581 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110e13c-052f-4978-9749-6052c52fdb9f None admin] PATH_INFO: `/v3/projects/0cd36903cbe141c6abc9a3df855737c6` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:57.244611 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110e13c-052f-4978-9749-6052c52fdb9f None admin] RBAC: Authorizing `identity:get_project(project_id=0cd36903cbe141c6abc9a3df855737c6)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:57.246274 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110e13c-052f-4978-9749-6052c52fdb9f None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:57.253246 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 548/1078] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:08:57 2022] GET /identity/v3/projects/0cd36903cbe141c6abc9a3df855737c6 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:57.254694 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110e13c-052f-4978-9749-6052c52fdb9f None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:57.256627 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefd094-f7cb-4433-b72b-8460c3bbebf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:57.257602 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefd094-f7cb-4433-b72b-8460c3bbebf6 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:57.257710 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefd094-f7cb-4433-b72b-8460c3bbebf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:57.258102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefd094-f7cb-4433-b72b-8460c3bbebf6 None admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:57.259677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cefd094-f7cb-4433-b72b-8460c3bbebf6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:57.259677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cefd094-f7cb-4433-b72b-8460c3bbebf6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:57.260579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cefd094-f7cb-4433-b72b-8460c3bbebf6 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:57.274374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 531/1079] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:08:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:08:57.281100 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefd094-f7cb-4433-b72b-8460c3bbebf6 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:57.293746 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:57.363890 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110e13c-052f-4978-9749-6052c52fdb9f None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:57.391904 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefd094-f7cb-4433-b72b-8460c3bbebf6 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:57.398897 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d35a92-96d7-4934-bef4-cdd3103e8efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:57.399594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d35a92-96d7-4934-bef4-cdd3103e8efc None admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:57.399873 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d35a92-96d7-4934-bef4-cdd3103e8efc None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:57.400144 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d35a92-96d7-4934-bef4-cdd3103e8efc None admin] PATH_INFO: `/v3/projects/0cd36903cbe141c6abc9a3df855737c6/users/3fef4ee8dfd34241a8551e2bbcca7048/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:57.402026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d35a92-96d7-4934-bef4-cdd3103e8efc None admin] RBAC: Authorizing `identity:create_grant(project_id=0cd36903cbe141c6abc9a3df855737c6, user_id=3fef4ee8dfd34241a8551e2bbcca7048, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:57.403488 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d35a92-96d7-4934-bef4-cdd3103e8efc None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:57.412558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:57.417923 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 549/1080] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 31 00:08:57 2022] PUT /identity/v3/projects/0cd36903cbe141c6abc9a3df855737c6/users/3fef4ee8dfd34241a8551e2bbcca7048/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:08:57.420784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d35a92-96d7-4934-bef4-cdd3103e8efc None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:57.431145 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fefb28-ca83-49b0-8429-6ccd53211ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:57.431873 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fefb28-ca83-49b0-8429-6ccd53211ee4 None admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:57.432160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fefb28-ca83-49b0-8429-6ccd53211ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:57.432441 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fefb28-ca83-49b0-8429-6ccd53211ee4 None admin] PATH_INFO: `/v3/projects/cd1e6daca4264a0cbc11a9160d1213eb/users/3aa53ae5a30146f69d7d6eefe4290dba/roles/39f496940d6844c8a43185fbd5fc1d21` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:57.434849 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fefb28-ca83-49b0-8429-6ccd53211ee4 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd1e6daca4264a0cbc11a9160d1213eb, user_id=3aa53ae5a30146f69d7d6eefe4290dba, role_id=39f496940d6844c8a43185fbd5fc1d21)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:57.438070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fefb28-ca83-49b0-8429-6ccd53211ee4 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:57.455140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 532/1081] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 31 00:08:57 2022] PUT /identity/v3/projects/cd1e6daca4264a0cbc11a9160d1213eb/users/3aa53ae5a30146f69d7d6eefe4290dba/roles/39f496940d6844c8a43185fbd5fc1d21 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:08:57.551716 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d35a92-96d7-4934-bef4-cdd3103e8efc None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:57.599521 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d3a16-62a5-4a89-a42d-e8e3a8f151f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:57.600170 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d3a16-62a5-4a89-a42d-e8e3a8f151f8 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:57.600423 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d3a16-62a5-4a89-a42d-e8e3a8f151f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:57.600695 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d3a16-62a5-4a89-a42d-e8e3a8f151f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:57.677806 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:57.680263 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d3a16-62a5-4a89-a42d-e8e3a8f151f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:57.681951 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d3a16-62a5-4a89-a42d-e8e3a8f151f8 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:57.788656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 550/1082] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:08:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:08:57.795417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fefb28-ca83-49b0-8429-6ccd53211ee4 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:57.866385 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:57.907764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fefb28-ca83-49b0-8429-6ccd53211ee4 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:57.943395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b357b-2043-4bfe-bf1a-8c4710ccfe2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:57.944035 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b357b-2043-4bfe-bf1a-8c4710ccfe2d None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:57.944313 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b357b-2043-4bfe-bf1a-8c4710ccfe2d None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:57.944585 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b357b-2043-4bfe-bf1a-8c4710ccfe2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:57.956547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b357b-2043-4bfe-bf1a-8c4710ccfe2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:57.958011 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b357b-2043-4bfe-bf1a-8c4710ccfe2d None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:57.978062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-3c3b357b-2043-4bfe-bf1a-8c4710ccfe2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:57.978062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:57.978062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:57.978062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:57.978062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:57.978062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:57.978062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:57.978062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:57.978062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:57.978062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:57.978062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:57.978062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:57.978062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:08:57.978062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:08:57.978062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:08:57.978062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:08:57.979072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:57.979072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:57.979072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:57.979072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:57.979072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:57.979072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:57.979072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:57.979072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:57.979072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:57.979072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:57.979072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:57.979072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:57.979072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:57.979072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:57.979072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:57.979072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:57.979072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:08:57.980008 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:08:57.980008 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:08:57.980008 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:57.983407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 533/1083] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:08:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:57.986187 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d3a16-62a5-4a89-a42d-e8e3a8f151f8 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:58.086442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:58.090345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d3a16-62a5-4a89-a42d-e8e3a8f151f8 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:58.127516 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf08d40-fb7f-410e-962a-a23e5ac46eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:58.128053 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf08d40-fb7f-410e-962a-a23e5ac46eca None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:58.128334 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf08d40-fb7f-410e-962a-a23e5ac46eca None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:58.128605 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf08d40-fb7f-410e-962a-a23e5ac46eca None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:58.129249 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf08d40-fb7f-410e-962a-a23e5ac46eca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:08:58.129554 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf08d40-fb7f-410e-962a-a23e5ac46eca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:08:58.130921 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf08d40-fb7f-410e-962a-a23e5ac46eca None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:08:58.143250 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 551/1084] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:08:59.367686 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b9755-9d5d-4f42-8526-d7f2d1664e8e None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:59.368213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b9755-9d5d-4f42-8526-d7f2d1664e8e None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:59.368213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b9755-9d5d-4f42-8526-d7f2d1664e8e None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:59.369577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 534/1085] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:59.375700 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa9df9-c815-4dbf-a1c5-4343edce3e20 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:59.376117 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa9df9-c815-4dbf-a1c5-4343edce3e20 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:59.376225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa9df9-c815-4dbf-a1c5-4343edce3e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:59.394331 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-ecfa9df9-c815-4dbf-a1c5-4343edce3e20 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:59.395540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-ecfa9df9-c815-4dbf-a1c5-4343edce3e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:59.412474 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f310e35-5632-4681-80bb-861dcd266574 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:59.412757 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f310e35-5632-4681-80bb-861dcd266574 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:59.413031 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f310e35-5632-4681-80bb-861dcd266574 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:59.414364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 535/1086] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:08:59.419007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c0654-c229-4b13-bc4c-5e207a9ab3ae None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:59.419318 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c0654-c229-4b13-bc4c-5e207a9ab3ae None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:59.420369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c0654-c229-4b13-bc4c-5e207a9ab3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:59.438188 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-963c0654-c229-4b13-bc4c-5e207a9ab3ae None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:08:59.439427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-963c0654-c229-4b13-bc4c-5e207a9ab3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:08:59.463798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 552/1087] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:08:59.510016 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 536/1088] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:08:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:08:59.738924 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa9df9-c815-4dbf-a1c5-4343edce3e20 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:59.781407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c0654-c229-4b13-bc4c-5e207a9ab3ae None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:59.831310 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:59.833595 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:08:59.841328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa9df9-c815-4dbf-a1c5-4343edce3e20 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:59.880650 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623399b-f54d-456f-8aa5-76bc1517d492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:59.881291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623399b-f54d-456f-8aa5-76bc1517d492 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:59.881588 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623399b-f54d-456f-8aa5-76bc1517d492 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:59.881839 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623399b-f54d-456f-8aa5-76bc1517d492 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:59.887660 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c0654-c229-4b13-bc4c-5e207a9ab3ae None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:08:59.897268 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623399b-f54d-456f-8aa5-76bc1517d492 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:08:59.902110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-0623399b-f54d-456f-8aa5-76bc1517d492 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:08:59.902110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:59.902110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:59.902110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:59.902110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:59.902110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:59.902110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:59.902110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:59.902110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:59.902110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:59.902110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:59.902110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:59.902110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:08:59.902110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:08:59.902110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:08:59.902110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:08:59.903085 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:08:59.903085 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:08:59.903085 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:08:59.903085 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:08:59.903085 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:59.903085 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:59.903085 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:59.903085 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:59.903085 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:59.903085 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:59.903085 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:59.903085 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:59.903085 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:59.903085 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:59.903085 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:59.903085 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:59.904505 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:59.904505 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:59.904505 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:59.904505 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:59.904505 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:08:59.904505 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:08:59.904505 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:08:59.904505 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:08:59.904505 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:08:59.904505 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:08:59.904505 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:08:59.904505 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:08:59.904505 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 553/1089] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:08:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:08:59.910154 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623399b-f54d-456f-8aa5-76bc1517d492 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:08:59.929587 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d1093-bc64-4158-ade3-1e7034064200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:08:59.930231 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d1093-bc64-4158-ade3-1e7034064200 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:08:59.930502 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d1093-bc64-4158-ade3-1e7034064200 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:08:59.930770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d1093-bc64-4158-ade3-1e7034064200 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:08:59.940869 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d1093-bc64-4158-ade3-1e7034064200 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:08:59.945568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-897d1093-bc64-4158-ade3-1e7034064200 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 00:08:59.945568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:08:59.945568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:08:59.945568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:08:59.945568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:08:59.945568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:08:59.945568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:08:59.945568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:08:59.945568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:08:59.945568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:08:59.945568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:08:59.945568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:08:59.945568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:08:59.945568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:08:59.945568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:08:59.945568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:08:59.946550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:08:59.946550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:08:59.946550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:08:59.946550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:08:59.946550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:08:59.946550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:08:59.946550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:08:59.946550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:08:59.946550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:08:59.946550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:08:59.946550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:08:59.946550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:08:59.946550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:08:59.946550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:08:59.946550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:08:59.946550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:08:59.947469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:08:59.947469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:08:59.947469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:08:59.947469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:08:59.947469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:08:59.947469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:08:59.947469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:08:59.947469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:08:59.947469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:08:59.947469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:08:59.947469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 00:08:59.947469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:08:59.947469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 537/1090] 10.222.0.67 () {56 vars in 1120 bytes} [Wed Aug 31 00:08:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:08:59.952867 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d1093-bc64-4158-ade3-1e7034064200 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:00.016405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623399b-f54d-456f-8aa5-76bc1517d492 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:00.050881 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c5723-f2d4-4adb-8cd0-69b7cb42fd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:00.051515 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c5723-f2d4-4adb-8cd0-69b7cb42fd24 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:00.051793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c5723-f2d4-4adb-8cd0-69b7cb42fd24 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:00.052064 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c5723-f2d4-4adb-8cd0-69b7cb42fd24 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:00.052724 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c5723-f2d4-4adb-8cd0-69b7cb42fd24 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:09:00.053024 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c5723-f2d4-4adb-8cd0-69b7cb42fd24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:00.054401 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c5723-f2d4-4adb-8cd0-69b7cb42fd24 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:00.058978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d1093-bc64-4158-ade3-1e7034064200 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:00.064170 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 554/1091] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:08:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:09:00.069808 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c5723-f2d4-4adb-8cd0-69b7cb42fd24 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:00.069977 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:00.084825 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:00.095028 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4dc2f4-1fc8-4db9-989e-45cead6dd14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:00.095658 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4dc2f4-1fc8-4db9-989e-45cead6dd14a None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:00.095935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4dc2f4-1fc8-4db9-989e-45cead6dd14a None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:00.096201 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4dc2f4-1fc8-4db9-989e-45cead6dd14a None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:00.096860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4dc2f4-1fc8-4db9-989e-45cead6dd14a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:09:00.097159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4dc2f4-1fc8-4db9-989e-45cead6dd14a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:00.098515 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4dc2f4-1fc8-4db9-989e-45cead6dd14a None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:00.106652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 538/1092] 10.222.0.67 () {56 vars in 1115 bytes} [Wed Aug 31 00:08:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:00.113018 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4dc2f4-1fc8-4db9-989e-45cead6dd14a None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:00.169144 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c5723-f2d4-4adb-8cd0-69b7cb42fd24 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:00.205445 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51356a42-453e-4edc-b62d-baab28f71692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:00.206500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51356a42-453e-4edc-b62d-baab28f71692 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:00.206963 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51356a42-453e-4edc-b62d-baab28f71692 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:00.207445 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51356a42-453e-4edc-b62d-baab28f71692 None admin] PATH_INFO: `/v3/users/3fef4ee8dfd34241a8551e2bbcca7048` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:00.209134 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51356a42-453e-4edc-b62d-baab28f71692 None admin] RBAC: Authorizing `identity:get_user(user_id=3fef4ee8dfd34241a8551e2bbcca7048)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:00.211069 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51356a42-453e-4edc-b62d-baab28f71692 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:00.212211 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 555/1093] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:09:00 2022] GET /identity/v3/users/3fef4ee8dfd34241a8551e2bbcca7048 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:09:00.218672 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51356a42-453e-4edc-b62d-baab28f71692 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:00.220538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4dc2f4-1fc8-4db9-989e-45cead6dd14a None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:00.259146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f80cc-aa18-4e40-836c-c158bea783e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:00.259750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f80cc-aa18-4e40-836c-c158bea783e3 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:00.260030 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f80cc-aa18-4e40-836c-c158bea783e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:00.260304 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f80cc-aa18-4e40-836c-c158bea783e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:00.272222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f80cc-aa18-4e40-836c-c158bea783e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:00.273844 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f80cc-aa18-4e40-836c-c158bea783e3 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:00.288930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-1d5f80cc-aa18-4e40-836c-c158bea783e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:09:00.288930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:09:00.288930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:09:00.288930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:09:00.288930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:09:00.288930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:09:00.288930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:09:00.288930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:09:00.288930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:09:00.288930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:09:00.288930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:09:00.288930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:09:00.288930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:09:00.288930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:09:00.288930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:09:00.288930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:09:00.289912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:09:00.289912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:09:00.289912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:09:00.289912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:09:00.289912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:09:00.289912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:09:00.289912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:09:00.289912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:09:00.289912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:09:00.289912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:09:00.289912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:09:00.289912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:09:00.289912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:09:00.289912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:09:00.289912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:09:00.289912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:09:00.289912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:09:00.290797 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:09:00.290797 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:09:00.290797 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:09:00.290797 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 539/1094] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:09:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:09:00.296761 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f80cc-aa18-4e40-836c-c158bea783e3 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:00.308199 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:00.324285 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:00.331328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51356a42-453e-4edc-b62d-baab28f71692 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:00.365333 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63283d45-5b80-4a59-98ea-b5eef1bb4ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:00.365939 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63283d45-5b80-4a59-98ea-b5eef1bb4ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:00.366218 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63283d45-5b80-4a59-98ea-b5eef1bb4ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:00.366494 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63283d45-5b80-4a59-98ea-b5eef1bb4ed9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:00.400086 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f80cc-aa18-4e40-836c-c158bea783e3 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:00.435867 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fcf20a-f618-42c2-8bff-196c86239fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:00.436470 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63283d45-5b80-4a59-98ea-b5eef1bb4ed9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:00.436880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fcf20a-f618-42c2-8bff-196c86239fae None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:00.436880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fcf20a-f618-42c2-8bff-196c86239fae None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:00.437081 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fcf20a-f618-42c2-8bff-196c86239fae None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:00.437305 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63283d45-5b80-4a59-98ea-b5eef1bb4ed9 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:00.437720 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fcf20a-f618-42c2-8bff-196c86239fae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:09:00.438024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fcf20a-f618-42c2-8bff-196c86239fae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:00.439391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fcf20a-f618-42c2-8bff-196c86239fae None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:00.452223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 540/1095] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:09:00.457402 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fcf20a-f618-42c2-8bff-196c86239fae None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:00.522479 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:00.542260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 556/1096] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:09:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 00:09:00.542530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:00.549692 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63283d45-5b80-4a59-98ea-b5eef1bb4ed9 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:00.558317 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fcf20a-f618-42c2-8bff-196c86239fae None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:00.598055 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c057cf-a95a-43c5-bcd9-661734225edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:00.598689 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c057cf-a95a-43c5-bcd9-661734225edb None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:00.598964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c057cf-a95a-43c5-bcd9-661734225edb None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:00.599267 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c057cf-a95a-43c5-bcd9-661734225edb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:00.606547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c057cf-a95a-43c5-bcd9-661734225edb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:00.608869 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c057cf-a95a-43c5-bcd9-661734225edb None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:00.620914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-c8c057cf-a95a-43c5-bcd9-661734225edb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:09:00.620914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:09:00.620914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:09:00.620914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:09:00.620914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:09:00.620914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:09:00.620914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:09:00.620914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:09:00.620914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:09:00.620914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:09:00.620914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:09:00.620914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:09:00.620914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:09:00.620914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:09:00.620914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:09:00.620914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:09:00.622066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:09:00.622066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:09:00.622066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:09:00.622066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:09:00.622066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:09:00.622066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:09:00.622066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:09:00.622066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:09:00.622066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:09:00.622066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:09:00.622066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:09:00.622066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:09:00.622066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:09:00.622066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:09:00.622066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:09:00.622066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:09:00.622066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:09:00.622066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:09:00.624292 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:09:00.624292 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:09:00.624292 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:09:00.624292 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:09:00.624292 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:09:00.624292 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:09:00.624292 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:09:00.624292 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:09:00.624292 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 00:09:00.624292 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:09:00.624292 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 541/1097] 10.222.0.67 () {56 vars in 1093 bytes} [Wed Aug 31 00:09:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:09:00.628712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c057cf-a95a-43c5-bcd9-661734225edb None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:00.657265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63283d45-5b80-4a59-98ea-b5eef1bb4ed9 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:00.691564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ae0ed-06d7-408b-8562-471bd79e69e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:00.692188 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ae0ed-06d7-408b-8562-471bd79e69e3 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:00.692462 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ae0ed-06d7-408b-8562-471bd79e69e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:00.692727 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ae0ed-06d7-408b-8562-471bd79e69e3 None admin] PATH_INFO: `/v3/projects/0cd36903cbe141c6abc9a3df855737c6` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:00.693952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295ae0ed-06d7-408b-8562-471bd79e69e3 None admin] RBAC: Authorizing `identity:get_project(project_id=0cd36903cbe141c6abc9a3df855737c6)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:00.695404 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295ae0ed-06d7-408b-8562-471bd79e69e3 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:00.697016 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 557/1098] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:09:00 2022] GET /identity/v3/projects/0cd36903cbe141c6abc9a3df855737c6 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:09:00.703302 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ae0ed-06d7-408b-8562-471bd79e69e3 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:00.729525 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c057cf-a95a-43c5-bcd9-661734225edb None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:00.752242 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:00.763000 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445e4b3-a5e4-4e11-8194-b5238fbcc79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:00.763733 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445e4b3-a5e4-4e11-8194-b5238fbcc79d None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:00.764012 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445e4b3-a5e4-4e11-8194-b5238fbcc79d None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:00.764282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445e4b3-a5e4-4e11-8194-b5238fbcc79d None admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:00.765005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2445e4b3-a5e4-4e11-8194-b5238fbcc79d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:09:00.765309 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2445e4b3-a5e4-4e11-8194-b5238fbcc79d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:00.766727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2445e4b3-a5e4-4e11-8194-b5238fbcc79d None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:00.784175 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 542/1099] 10.222.0.67 () {56 vars in 1133 bytes} [Wed Aug 31 00:09:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:09:00.789954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445e4b3-a5e4-4e11-8194-b5238fbcc79d None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:00.792572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:00.804149 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ae0ed-06d7-408b-8562-471bd79e69e3 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:00.838395 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee657ab-c88e-4343-9c65-64c7e76af55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:00.838995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee657ab-c88e-4343-9c65-64c7e76af55c None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:00.839302 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee657ab-c88e-4343-9c65-64c7e76af55c None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:00.839571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee657ab-c88e-4343-9c65-64c7e76af55c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:00.840277 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee657ab-c88e-4343-9c65-64c7e76af55c None admin] RBAC: Adding query filter params (role.id=34df06ce84ad4cf8b3bd1a4490228017, scope.project.id=0cd36903cbe141c6abc9a3df855737c6, user.id=3fef4ee8dfd34241a8551e2bbcca7048) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:09:00.840583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee657ab-c88e-4343-9c65-64c7e76af55c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:00.842616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee657ab-c88e-4343-9c65-64c7e76af55c None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:00.848470 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 558/1100] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:09:00 2022] GET /identity/v3/role_assignments?user.id=3fef4ee8dfd34241a8551e2bbcca7048&scope.project.id=0cd36903cbe141c6abc9a3df855737c6&role.id=34df06ce84ad4cf8b3bd1a4490228017 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:09:00.896072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445e4b3-a5e4-4e11-8194-b5238fbcc79d None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:00.932839 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ec974-fe3f-4a76-a634-2feded480185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:00.933453 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ec974-fe3f-4a76-a634-2feded480185 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:00.933725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ec974-fe3f-4a76-a634-2feded480185 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:00.933990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ec974-fe3f-4a76-a634-2feded480185 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:01.006349 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ec974-fe3f-4a76-a634-2feded480185 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:01.007823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ec974-fe3f-4a76-a634-2feded480185 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:01.009680 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:01.107729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 543/1101] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:09:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 00:09:01.113263 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee657ab-c88e-4343-9c65-64c7e76af55c None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:01.212260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee657ab-c88e-4343-9c65-64c7e76af55c None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:01.225512 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:01.246873 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b105f-7f68-42d0-bc57-b5826aa0af24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:01.247539 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b105f-7f68-42d0-bc57-b5826aa0af24 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:01.248433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b105f-7f68-42d0-bc57-b5826aa0af24 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:01.248716 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b105f-7f68-42d0-bc57-b5826aa0af24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:01.259264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b105f-7f68-42d0-bc57-b5826aa0af24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:01.260704 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b105f-7f68-42d0-bc57-b5826aa0af24 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:01.275541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-2d3b105f-7f68-42d0-bc57-b5826aa0af24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:09:01.275541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:09:01.275541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:09:01.275541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:09:01.275541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:09:01.275541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:09:01.275541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:09:01.275541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:09:01.275541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:09:01.275541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:09:01.275541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:09:01.275541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:09:01.275541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:09:01.275541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:09:01.275541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:09:01.275541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:09:01.276497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:09:01.276497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:09:01.276497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:09:01.276497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:09:01.276497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:09:01.276497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:09:01.276497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:09:01.276497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:09:01.276497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:09:01.276497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:09:01.276497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:09:01.276497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:09:01.276497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:09:01.276497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:09:01.276497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:09:01.276497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:09:01.276497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:09:01.277739 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:09:01.277739 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:09:01.277739 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:09:01.277739 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 559/1102] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:09:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:09:01.283416 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ec974-fe3f-4a76-a634-2feded480185 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:01.383324 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ec974-fe3f-4a76-a634-2feded480185 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:01.384777 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:01.418582 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f691f2-4ab8-4923-baf9-014c10201db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:01.419221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f691f2-4ab8-4923-baf9-014c10201db1 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:01.419628 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f691f2-4ab8-4923-baf9-014c10201db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:01.419910 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f691f2-4ab8-4923-baf9-014c10201db1 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:01.420544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f691f2-4ab8-4923-baf9-014c10201db1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:09:01.420846 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f691f2-4ab8-4923-baf9-014c10201db1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:01.422198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f691f2-4ab8-4923-baf9-014c10201db1 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:01.433421 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 544/1103] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:09:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:01.439650 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b105f-7f68-42d0-bc57-b5826aa0af24 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:01.539173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b105f-7f68-42d0-bc57-b5826aa0af24 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:01.572298 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18285d7-df47-4c2a-a561-f6eb628d0898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:01.573000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18285d7-df47-4c2a-a561-f6eb628d0898 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:01.573277 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18285d7-df47-4c2a-a561-f6eb628d0898 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:01.573548 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18285d7-df47-4c2a-a561-f6eb628d0898 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:01.580745 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18285d7-df47-4c2a-a561-f6eb628d0898 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:01.582192 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18285d7-df47-4c2a-a561-f6eb628d0898 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:01.596840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-b18285d7-df47-4c2a-a561-f6eb628d0898 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:09:01.596840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:09:01.596840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:09:01.596840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:09:01.596840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:09:01.596840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:09:01.596840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:09:01.596840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:09:01.596840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:09:01.596840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:09:01.596840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:09:01.596840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:09:01.596840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:09:01.596840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:09:01.596840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:09:01.597798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:09:01.597798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:09:01.597798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:09:01.597798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:09:01.597798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:09:01.597798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:09:01.597798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:09:01.597798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:09:01.597798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:09:01.597798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:09:01.597798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:09:01.597798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:09:01.597798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:09:01.597798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:09:01.597798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:09:01.597798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:09:01.597798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:09:01.598804 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:09:01.598804 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:09:01.598804 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:09:01.598804 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:09:01.598804 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:09:01.598804 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:09:01.598804 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:09:01.598804 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:09:01.598804 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 560/1104] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:09:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:09:01.603956 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f691f2-4ab8-4923-baf9-014c10201db1 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:01.706014 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f691f2-4ab8-4923-baf9-014c10201db1 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:01.739293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9bfc4d-9b86-43ab-9ce9-389e45ff27b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:01.739898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9bfc4d-9b86-43ab-9ce9-389e45ff27b0 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:01.740177 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9bfc4d-9b86-43ab-9ce9-389e45ff27b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:01.740450 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9bfc4d-9b86-43ab-9ce9-389e45ff27b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:01.741142 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9bfc4d-9b86-43ab-9ce9-389e45ff27b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:09:01.741448 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9bfc4d-9b86-43ab-9ce9-389e45ff27b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:01.742866 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9bfc4d-9b86-43ab-9ce9-389e45ff27b0 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:01.755101 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 545/1105] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:09:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:01.761298 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18285d7-df47-4c2a-a561-f6eb628d0898 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:01.787770 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:01.864132 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18285d7-df47-4c2a-a561-f6eb628d0898 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:01.899047 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf112a-a572-4c69-ad72-04f3d623fa85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:01.899663 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf112a-a572-4c69-ad72-04f3d623fa85 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:01.899944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf112a-a572-4c69-ad72-04f3d623fa85 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:01.900216 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf112a-a572-4c69-ad72-04f3d623fa85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:01.900944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf112a-a572-4c69-ad72-04f3d623fa85 None admin] RBAC: Adding query filter params (role.id=39f496940d6844c8a43185fbd5fc1d21, scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb, user.id=3aa53ae5a30146f69d7d6eefe4290dba) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:09:01.901251 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf112a-a572-4c69-ad72-04f3d623fa85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:01.902671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf112a-a572-4c69-ad72-04f3d623fa85 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:01.909476 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 561/1106] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:09:01 2022] GET /identity/v3/role_assignments?user.id=3aa53ae5a30146f69d7d6eefe4290dba&scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb&role.id=39f496940d6844c8a43185fbd5fc1d21 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:02.791912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97661257-65d9-4ff2-9c6f-f649f6671aaf None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:02.792531 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97661257-65d9-4ff2-9c6f-f649f6671aaf None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:02.792531 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97661257-65d9-4ff2-9c6f-f649f6671aaf None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:02.793804 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 546/1107] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:09:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:09:02.799222 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1434b19-69b6-4150-bd5e-3dada0a1bb2e None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:02.799669 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1434b19-69b6-4150-bd5e-3dada0a1bb2e None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:02.799769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1434b19-69b6-4150-bd5e-3dada0a1bb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:02.817943 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-a1434b19-69b6-4150-bd5e-3dada0a1bb2e None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:09:02.819131 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a1434b19-69b6-4150-bd5e-3dada0a1bb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:09:02.885616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 562/1108] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:09:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:09:03.162376 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97661257-65d9-4ff2-9c6f-f649f6671aaf None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:03.187775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:03.265062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97661257-65d9-4ff2-9c6f-f649f6671aaf None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:03.298414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94803a4b-1ee4-403f-aea9-eb7fd5c8e80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:03.299030 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94803a4b-1ee4-403f-aea9-eb7fd5c8e80c None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:03.299342 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94803a4b-1ee4-403f-aea9-eb7fd5c8e80c None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:03.299616 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94803a4b-1ee4-403f-aea9-eb7fd5c8e80c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:03.300713 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94803a4b-1ee4-403f-aea9-eb7fd5c8e80c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:03.302131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94803a4b-1ee4-403f-aea9-eb7fd5c8e80c None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:03.303720 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 547/1109] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:09:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:03.309474 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1434b19-69b6-4150-bd5e-3dada0a1bb2e None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:03.327851 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:03.406502 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1434b19-69b6-4150-bd5e-3dada0a1bb2e None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:03.438258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1845f9-1ea5-4a40-8297-e3a093672979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:03.438791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1845f9-1ea5-4a40-8297-e3a093672979 None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:03.439096 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1845f9-1ea5-4a40-8297-e3a093672979 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:03.439369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1845f9-1ea5-4a40-8297-e3a093672979 None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:03.440015 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1845f9-1ea5-4a40-8297-e3a093672979 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:03.441422 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1845f9-1ea5-4a40-8297-e3a093672979 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:03.463047 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 563/1110] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 31 00:09:03 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:09:05.439101 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dd583-3d57-4f46-8a99-4f5f52544852 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:05.439643 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dd583-3d57-4f46-8a99-4f5f52544852 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:05.439643 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dd583-3d57-4f46-8a99-4f5f52544852 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:05.440970 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 548/1111] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:09:05.446038 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52dfc3-1e47-4c30-9b1a-8c2d85bab909 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:05.446466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52dfc3-1e47-4c30-9b1a-8c2d85bab909 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:05.446594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52dfc3-1e47-4c30-9b1a-8c2d85bab909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:05.465121 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-8c52dfc3-1e47-4c30-9b1a-8c2d85bab909 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:09:05.466299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-8c52dfc3-1e47-4c30-9b1a-8c2d85bab909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:09:05.531955 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 564/1112] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:09:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:09:05.799673 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9dd583-3d57-4f46-8a99-4f5f52544852 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:05.902370 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:05.913442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9dd583-3d57-4f46-8a99-4f5f52544852 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:05.948024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830a391-8811-423f-9e79-c9012223075c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:05.948650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830a391-8811-423f-9e79-c9012223075c None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:05.948932 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830a391-8811-423f-9e79-c9012223075c None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:05.949207 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830a391-8811-423f-9e79-c9012223075c None admin] PATH_INFO: `/v3/roles/4d91c00f40304b9e9ef4e6c89f46e5c5` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:05.950598 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3830a391-8811-423f-9e79-c9012223075c None admin] RBAC: Authorizing `identity:get_role(role_id=4d91c00f40304b9e9ef4e6c89f46e5c5)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:05.952891 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3830a391-8811-423f-9e79-c9012223075c None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:05.954220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 549/1113] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:09:05 2022] GET /identity/v3/roles/4d91c00f40304b9e9ef4e6c89f46e5c5 => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:05.960063 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52dfc3-1e47-4c30-9b1a-8c2d85bab909 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:05.963155 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:06.060968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52dfc3-1e47-4c30-9b1a-8c2d85bab909 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:06.094027 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b175b7-c085-4f95-8afe-3db125eeb564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:06.094656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b175b7-c085-4f95-8afe-3db125eeb564 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:06.094937 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b175b7-c085-4f95-8afe-3db125eeb564 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:06.095242 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b175b7-c085-4f95-8afe-3db125eeb564 None admin] PATH_INFO: `/v3/users/5113a27d1f94478bb87bb8fb74ed3448` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:06.108498 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b175b7-c085-4f95-8afe-3db125eeb564 None admin] RBAC: Authorizing `identity:get_user(user_id=5113a27d1f94478bb87bb8fb74ed3448)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:06.109942 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b175b7-c085-4f95-8afe-3db125eeb564 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:06.111590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 565/1114] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:09:05 2022] GET /identity/v3/users/5113a27d1f94478bb87bb8fb74ed3448 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:06.117358 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830a391-8811-423f-9e79-c9012223075c None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:06.213999 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830a391-8811-423f-9e79-c9012223075c None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:06.246628 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c34c9b-e87d-4a1b-bdd2-0bd8640f5b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:06.247251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c34c9b-e87d-4a1b-bdd2-0bd8640f5b4a None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:06.247531 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c34c9b-e87d-4a1b-bdd2-0bd8640f5b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:06.248361 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c34c9b-e87d-4a1b-bdd2-0bd8640f5b4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:06.274564 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:06.317526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c34c9b-e87d-4a1b-bdd2-0bd8640f5b4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:06.318897 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c34c9b-e87d-4a1b-bdd2-0bd8640f5b4a None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:06.413415 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 550/1115] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:09:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:09:06.419343 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b175b7-c085-4f95-8afe-3db125eeb564 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:06.478354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:06.517039 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b175b7-c085-4f95-8afe-3db125eeb564 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:06.550387 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96719bae-3996-4781-86d7-60d8b7a254b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:06.551010 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96719bae-3996-4781-86d7-60d8b7a254b4 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:06.551321 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96719bae-3996-4781-86d7-60d8b7a254b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:06.551593 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96719bae-3996-4781-86d7-60d8b7a254b4 None admin] PATH_INFO: `/v3/projects/0cd36903cbe141c6abc9a3df855737c6` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:06.552711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96719bae-3996-4781-86d7-60d8b7a254b4 None admin] RBAC: Authorizing `identity:get_project(project_id=0cd36903cbe141c6abc9a3df855737c6)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:06.554142 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96719bae-3996-4781-86d7-60d8b7a254b4 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:06.555804 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 566/1116] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:09:06 2022] GET /identity/v3/projects/0cd36903cbe141c6abc9a3df855737c6 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:06.562544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c34c9b-e87d-4a1b-bdd2-0bd8640f5b4a None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:06.618541 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:06.659830 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c34c9b-e87d-4a1b-bdd2-0bd8640f5b4a None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:06.692515 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c785f-5582-4f06-9415-4cbc19064b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:06.693111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c785f-5582-4f06-9415-4cbc19064b0c None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:06.693390 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c785f-5582-4f06-9415-4cbc19064b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:06.693659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c785f-5582-4f06-9415-4cbc19064b0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:06.694374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c785f-5582-4f06-9415-4cbc19064b0c None admin] RBAC: Adding query filter params (role.id=4d91c00f40304b9e9ef4e6c89f46e5c5, scope.project.id=0cd36903cbe141c6abc9a3df855737c6, user.id=5113a27d1f94478bb87bb8fb74ed3448) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:09:06.694680 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c785f-5582-4f06-9415-4cbc19064b0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:06.696123 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c785f-5582-4f06-9415-4cbc19064b0c None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:06.702348 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 551/1117] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:09:06 2022] GET /identity/v3/role_assignments?user.id=5113a27d1f94478bb87bb8fb74ed3448&scope.project.id=0cd36903cbe141c6abc9a3df855737c6&role.id=4d91c00f40304b9e9ef4e6c89f46e5c5 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:08.601257 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050f86b-6373-4f10-9e46-0450fad22d4b None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:08.601823 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050f86b-6373-4f10-9e46-0450fad22d4b None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:08.601823 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050f86b-6373-4f10-9e46-0450fad22d4b None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:08.603181 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 567/1118] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:09:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:09:08.608892 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6f4fc-a03f-45ed-85d1-fa58381aebbf None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:08.609322 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6f4fc-a03f-45ed-85d1-fa58381aebbf None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:08.609512 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6f4fc-a03f-45ed-85d1-fa58381aebbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:08.628091 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-c3d6f4fc-a03f-45ed-85d1-fa58381aebbf None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:09:08.629249 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c3d6f4fc-a03f-45ed-85d1-fa58381aebbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:09:08.694800 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 552/1119] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:09:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:09:08.981058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050f86b-6373-4f10-9e46-0450fad22d4b None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:09.081353 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050f86b-6373-4f10-9e46-0450fad22d4b None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:09.109881 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:09.115905 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81508dea-1919-458d-a239-7512e5f58f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:09.116530 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81508dea-1919-458d-a239-7512e5f58f92 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:09.116808 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81508dea-1919-458d-a239-7512e5f58f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:09.117074 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81508dea-1919-458d-a239-7512e5f58f92 None admin] PATH_INFO: `/v3/roles/4d91c00f40304b9e9ef4e6c89f46e5c5` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:09.118440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81508dea-1919-458d-a239-7512e5f58f92 None admin] RBAC: Authorizing `identity:get_role(role_id=4d91c00f40304b9e9ef4e6c89f46e5c5)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:09.119830 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81508dea-1919-458d-a239-7512e5f58f92 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:09.121143 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 568/1120] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:09:08 2022] GET /identity/v3/roles/4d91c00f40304b9e9ef4e6c89f46e5c5 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:09.127135 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d6f4fc-a03f-45ed-85d1-fa58381aebbf None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:09.177939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:09.223557 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d6f4fc-a03f-45ed-85d1-fa58381aebbf None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:09.255893 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff2679-d5c0-4ed3-9dc3-18e4af62b2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:09.256472 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff2679-d5c0-4ed3-9dc3-18e4af62b2ef None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:09.256747 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff2679-d5c0-4ed3-9dc3-18e4af62b2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:09.257014 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff2679-d5c0-4ed3-9dc3-18e4af62b2ef None admin] PATH_INFO: `/v3/users/5113a27d1f94478bb87bb8fb74ed3448` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:09.258115 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff2679-d5c0-4ed3-9dc3-18e4af62b2ef None admin] RBAC: Authorizing `identity:get_user(user_id=5113a27d1f94478bb87bb8fb74ed3448)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:09.259553 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff2679-d5c0-4ed3-9dc3-18e4af62b2ef None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:09.261108 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 553/1121] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:09:09 2022] GET /identity/v3/users/5113a27d1f94478bb87bb8fb74ed3448 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:09.267527 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81508dea-1919-458d-a239-7512e5f58f92 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:09.362799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81508dea-1919-458d-a239-7512e5f58f92 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:09.395925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9204d69-68de-4c6d-93cc-02985740703d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:09.396534 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9204d69-68de-4c6d-93cc-02985740703d None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:09.396808 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9204d69-68de-4c6d-93cc-02985740703d None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:09.397071 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9204d69-68de-4c6d-93cc-02985740703d None admin] PATH_INFO: `/v3/projects/0cd36903cbe141c6abc9a3df855737c6` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:09.398162 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9204d69-68de-4c6d-93cc-02985740703d None admin] RBAC: Authorizing `identity:get_project(project_id=0cd36903cbe141c6abc9a3df855737c6)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:09.399598 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9204d69-68de-4c6d-93cc-02985740703d None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:09.401177 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 569/1122] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:09:09 2022] GET /identity/v3/projects/0cd36903cbe141c6abc9a3df855737c6 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:09.407924 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff2679-d5c0-4ed3-9dc3-18e4af62b2ef None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:09.505631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff2679-d5c0-4ed3-9dc3-18e4af62b2ef None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:09.525471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:09.539321 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d70e14-8a8c-4620-a3df-b92f89899fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:09.539972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d70e14-8a8c-4620-a3df-b92f89899fe9 None admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:09.540314 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d70e14-8a8c-4620-a3df-b92f89899fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:09.540595 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d70e14-8a8c-4620-a3df-b92f89899fe9 None admin] PATH_INFO: `/v3/projects/0cd36903cbe141c6abc9a3df855737c6/users/5113a27d1f94478bb87bb8fb74ed3448/roles/4d91c00f40304b9e9ef4e6c89f46e5c5` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:09.542719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d70e14-8a8c-4620-a3df-b92f89899fe9 None admin] RBAC: Authorizing `identity:create_grant(project_id=0cd36903cbe141c6abc9a3df855737c6, user_id=5113a27d1f94478bb87bb8fb74ed3448, role_id=4d91c00f40304b9e9ef4e6c89f46e5c5)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:09.544176 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d70e14-8a8c-4620-a3df-b92f89899fe9 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:09.556296 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 554/1123] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 31 00:09:09 2022] PUT /identity/v3/projects/0cd36903cbe141c6abc9a3df855737c6/users/5113a27d1f94478bb87bb8fb74ed3448/roles/4d91c00f40304b9e9ef4e6c89f46e5c5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:09:11.428767 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c578b7c-2c30-4f40-9e30-7f79910d44b4 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:11.429288 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c578b7c-2c30-4f40-9e30-7f79910d44b4 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:11.429288 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c578b7c-2c30-4f40-9e30-7f79910d44b4 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:11.430620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 570/1124] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:09:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:09:11.435788 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64330e4-5cd5-4913-bccc-57723e87ea1d None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:11.436390 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64330e4-5cd5-4913-bccc-57723e87ea1d None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:11.436518 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64330e4-5cd5-4913-bccc-57723e87ea1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:11.454702 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-b64330e4-5cd5-4913-bccc-57723e87ea1d None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:09:11.455905 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b64330e4-5cd5-4913-bccc-57723e87ea1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:09:11.521026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 555/1125] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:09:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:09:11.788381 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c578b7c-2c30-4f40-9e30-7f79910d44b4 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:11.851864 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:11.887967 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c578b7c-2c30-4f40-9e30-7f79910d44b4 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:11.921827 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93aa9f8-0d07-49a2-8b09-888c2e5720a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:11.922446 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93aa9f8-0d07-49a2-8b09-888c2e5720a5 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:11.922726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93aa9f8-0d07-49a2-8b09-888c2e5720a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:11.923000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93aa9f8-0d07-49a2-8b09-888c2e5720a5 None admin] PATH_INFO: `/v3/roles/4d91c00f40304b9e9ef4e6c89f46e5c5` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:11.924431 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93aa9f8-0d07-49a2-8b09-888c2e5720a5 None admin] RBAC: Authorizing `identity:get_role(role_id=4d91c00f40304b9e9ef4e6c89f46e5c5)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:11.925794 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93aa9f8-0d07-49a2-8b09-888c2e5720a5 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:11.927128 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 571/1126] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:09:11 2022] GET /identity/v3/roles/4d91c00f40304b9e9ef4e6c89f46e5c5 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:11.932898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64330e4-5cd5-4913-bccc-57723e87ea1d None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:12.032165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64330e4-5cd5-4913-bccc-57723e87ea1d None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:12.042574 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:12.076812 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f248b8-cb4b-438f-a21e-54a5f2e6a48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:12.077560 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f248b8-cb4b-438f-a21e-54a5f2e6a48f None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:12.077843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f248b8-cb4b-438f-a21e-54a5f2e6a48f None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:12.078103 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f248b8-cb4b-438f-a21e-54a5f2e6a48f None admin] PATH_INFO: `/v3/users/5113a27d1f94478bb87bb8fb74ed3448` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:12.079262 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f248b8-cb4b-438f-a21e-54a5f2e6a48f None admin] RBAC: Authorizing `identity:get_user(user_id=5113a27d1f94478bb87bb8fb74ed3448)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:12.080674 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f248b8-cb4b-438f-a21e-54a5f2e6a48f None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:12.082262 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 556/1127] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:09:11 2022] GET /identity/v3/users/5113a27d1f94478bb87bb8fb74ed3448 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:12.088074 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93aa9f8-0d07-49a2-8b09-888c2e5720a5 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:12.184540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93aa9f8-0d07-49a2-8b09-888c2e5720a5 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:12.217816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1666c3-f3ba-4e4d-9295-38d0c076ac64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:12.218413 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1666c3-f3ba-4e4d-9295-38d0c076ac64 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:12.218691 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1666c3-f3ba-4e4d-9295-38d0c076ac64 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:12.218960 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1666c3-f3ba-4e4d-9295-38d0c076ac64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:12.221576 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:12.284828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1666c3-f3ba-4e4d-9295-38d0c076ac64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:12.286264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1666c3-f3ba-4e4d-9295-38d0c076ac64 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:12.379951 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 572/1128] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:09:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:09:12.385944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f248b8-cb4b-438f-a21e-54a5f2e6a48f None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:12.485191 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f248b8-cb4b-438f-a21e-54a5f2e6a48f None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:12.517961 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ba07f-5c09-4099-b259-dba31b2b2b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:12.518578 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ba07f-5c09-4099-b259-dba31b2b2b95 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:12.518857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ba07f-5c09-4099-b259-dba31b2b2b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:12.519164 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ba07f-5c09-4099-b259-dba31b2b2b95 None admin] PATH_INFO: `/v3/projects/0cd36903cbe141c6abc9a3df855737c6` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:12.520269 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6ba07f-5c09-4099-b259-dba31b2b2b95 None admin] RBAC: Authorizing `identity:get_project(project_id=0cd36903cbe141c6abc9a3df855737c6)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:12.521688 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6ba07f-5c09-4099-b259-dba31b2b2b95 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:12.523340 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 557/1129] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:09:12 2022] GET /identity/v3/projects/0cd36903cbe141c6abc9a3df855737c6 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:12.530349 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1666c3-f3ba-4e4d-9295-38d0c076ac64 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:12.563351 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:12.628635 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1666c3-f3ba-4e4d-9295-38d0c076ac64 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:12.661603 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94fc52a-18de-418e-8d76-e876d208867b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:12.662138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fc52a-18de-418e-8d76-e876d208867b None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:12.662416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fc52a-18de-418e-8d76-e876d208867b None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:12.662688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fc52a-18de-418e-8d76-e876d208867b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:12.663434 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94fc52a-18de-418e-8d76-e876d208867b None admin] RBAC: Adding query filter params (role.id=4d91c00f40304b9e9ef4e6c89f46e5c5, scope.project.id=0cd36903cbe141c6abc9a3df855737c6, user.id=5113a27d1f94478bb87bb8fb74ed3448) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:09:12.663742 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94fc52a-18de-418e-8d76-e876d208867b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:12.665159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94fc52a-18de-418e-8d76-e876d208867b None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:12.670869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 573/1130] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:09:12 2022] GET /identity/v3/role_assignments?user.id=5113a27d1f94478bb87bb8fb74ed3448&scope.project.id=0cd36903cbe141c6abc9a3df855737c6&role.id=4d91c00f40304b9e9ef4e6c89f46e5c5 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:14.608521 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c05e5-373b-4f11-a4c5-751c236775bf None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:14.609074 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c05e5-373b-4f11-a4c5-751c236775bf None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:14.609074 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c05e5-373b-4f11-a4c5-751c236775bf None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:14.610405 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 558/1131] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:09:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:09:14.615762 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b44761-6ec7-49ac-99f8-c4d854fc94d0 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:14.616154 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b44761-6ec7-49ac-99f8-c4d854fc94d0 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:14.616318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b44761-6ec7-49ac-99f8-c4d854fc94d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:14.635228 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-45b44761-6ec7-49ac-99f8-c4d854fc94d0 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:09:14.636402 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-45b44761-6ec7-49ac-99f8-c4d854fc94d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:09:14.709940 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 574/1132] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:09:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:09:14.985865 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c05e5-373b-4f11-a4c5-751c236775bf None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:15.039732 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:15.095178 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c05e5-373b-4f11-a4c5-751c236775bf None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:15.129446 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa4a453-13a8-4793-96a2-31c0edd61ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:15.130059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa4a453-13a8-4793-96a2-31c0edd61ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:15.130332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa4a453-13a8-4793-96a2-31c0edd61ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:15.130596 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa4a453-13a8-4793-96a2-31c0edd61ea9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:15.131698 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa4a453-13a8-4793-96a2-31c0edd61ea9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:15.133111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa4a453-13a8-4793-96a2-31c0edd61ea9 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:15.134651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 559/1133] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:09:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:15.140684 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b44761-6ec7-49ac-99f8-c4d854fc94d0 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:15.167360 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:15.239892 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b44761-6ec7-49ac-99f8-c4d854fc94d0 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:15.274008 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a7295-47c3-439b-abf2-50d66c557269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:15.274607 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a7295-47c3-439b-abf2-50d66c557269 None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:15.274885 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a7295-47c3-439b-abf2-50d66c557269 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:15.275188 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a7295-47c3-439b-abf2-50d66c557269 None admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:15.275841 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a7295-47c3-439b-abf2-50d66c557269 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:15.277263 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a7295-47c3-439b-abf2-50d66c557269 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:15.303375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 575/1134] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 31 00:09:15 2022] POST /identity/v3/projects => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:09:17.256503 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629c9cb-3459-42a9-9baa-1b07affd0838 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:17.257001 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629c9cb-3459-42a9-9baa-1b07affd0838 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:17.257001 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629c9cb-3459-42a9-9baa-1b07affd0838 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:17.258289 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 560/1135] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:09:17.263343 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da2eed-e15f-4ef5-a53f-88e8e003b5e7 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:17.263829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da2eed-e15f-4ef5-a53f-88e8e003b5e7 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:17.264131 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da2eed-e15f-4ef5-a53f-88e8e003b5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:17.282589 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-e0da2eed-e15f-4ef5-a53f-88e8e003b5e7 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:09:17.283791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e0da2eed-e15f-4ef5-a53f-88e8e003b5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:09:17.350149 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 576/1136] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:09:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:09:17.618043 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629c9cb-3459-42a9-9baa-1b07affd0838 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:17.718017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629c9cb-3459-42a9-9baa-1b07affd0838 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:17.731580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:17.751781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a807bff-fe4a-46d3-92d2-36e1523ad28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:17.752407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a807bff-fe4a-46d3-92d2-36e1523ad28f None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:17.752685 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a807bff-fe4a-46d3-92d2-36e1523ad28f None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:17.753209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a807bff-fe4a-46d3-92d2-36e1523ad28f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:17.754313 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a807bff-fe4a-46d3-92d2-36e1523ad28f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:17.755768 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a807bff-fe4a-46d3-92d2-36e1523ad28f None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:17.757334 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 561/1137] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:09:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:17.763046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da2eed-e15f-4ef5-a53f-88e8e003b5e7 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:17.775060 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:17.863187 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da2eed-e15f-4ef5-a53f-88e8e003b5e7 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:17.895835 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420f149-76c0-43be-8ae9-e50aff1962c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:17.896371 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420f149-76c0-43be-8ae9-e50aff1962c6 None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:17.896648 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420f149-76c0-43be-8ae9-e50aff1962c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:17.896916 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420f149-76c0-43be-8ae9-e50aff1962c6 None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:17.897564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420f149-76c0-43be-8ae9-e50aff1962c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:17.899098 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420f149-76c0-43be-8ae9-e50aff1962c6 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:17.920694 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 577/1138] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 31 00:09:17 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:09:19.868378 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6141e58-da9b-437f-9388-41c55d25c128 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:19.868932 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6141e58-da9b-437f-9388-41c55d25c128 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:19.868932 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6141e58-da9b-437f-9388-41c55d25c128 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:19.870301 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 562/1139] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:09:19.876395 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4e859-fbb6-4b46-b71f-068ee48d7a40 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:19.876789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4e859-fbb6-4b46-b71f-068ee48d7a40 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:19.876948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4e859-fbb6-4b46-b71f-068ee48d7a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:19.895375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-f1a4e859-fbb6-4b46-b71f-068ee48d7a40 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:09:19.896553 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f1a4e859-fbb6-4b46-b71f-068ee48d7a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:09:19.954449 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:19.962699 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 578/1140] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:09:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:09:20.230979 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6141e58-da9b-437f-9388-41c55d25c128 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:20.330434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6141e58-da9b-437f-9388-41c55d25c128 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:20.363505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad8e64-2878-4c78-ab6a-992dc2f48cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:20.364123 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad8e64-2878-4c78-ab6a-992dc2f48cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:20.364401 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad8e64-2878-4c78-ab6a-992dc2f48cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:20.364671 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad8e64-2878-4c78-ab6a-992dc2f48cf6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:20.374989 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad8e64-2878-4c78-ab6a-992dc2f48cf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:09:20.380009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-f2ad8e64-2878-4c78-ab6a-992dc2f48cf6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:09:20.380009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:09:20.380009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:09:20.380009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:09:20.380009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:09:20.380009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:09:20.380009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:09:20.380009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:09:20.380009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:09:20.380009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:09:20.380009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:09:20.380009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:09:20.380009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:09:20.380009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:09:20.380009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:09:20.380009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:09:20.381181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:09:20.381181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:09:20.381181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:09:20.381181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:09:20.381181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:09:20.381181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:09:20.381181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:09:20.381181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:09:20.381181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:09:20.381181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:09:20.381181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:09:20.381181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:09:20.381181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:09:20.381181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:09:20.381181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:09:20.381181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:09:20.382149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:09:20.382149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:09:20.382149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:09:20.382149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:09:20.382149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:09:20.382149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:09:20.382149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:09:20.382149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:09:20.382149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:09:20.382149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:09:20.382149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:09:20.382149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:09:20.382149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 563/1141] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:09:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:09:20.387185 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4e859-fbb6-4b46-b71f-068ee48d7a40 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:20.483702 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4e859-fbb6-4b46-b71f-068ee48d7a40 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:20.517410 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1216455-f640-4c11-bfe2-01b584544f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:20.518009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1216455-f640-4c11-bfe2-01b584544f39 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:20.518286 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1216455-f640-4c11-bfe2-01b584544f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:20.518554 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1216455-f640-4c11-bfe2-01b584544f39 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:20.519234 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1216455-f640-4c11-bfe2-01b584544f39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:09:20.519535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1216455-f640-4c11-bfe2-01b584544f39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:20.520890 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1216455-f640-4c11-bfe2-01b584544f39 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:20.528683 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 579/1142] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:09:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:20.535231 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad8e64-2878-4c78-ab6a-992dc2f48cf6 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:20.577385 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:20.633008 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad8e64-2878-4c78-ab6a-992dc2f48cf6 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:20.665250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5138da-9014-45fd-b799-bc366f613c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:20.665861 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5138da-9014-45fd-b799-bc366f613c0c None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:20.666140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5138da-9014-45fd-b799-bc366f613c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:20.666411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5138da-9014-45fd-b799-bc366f613c0c None admin] PATH_INFO: `/v3/users/e8f8904939e24284be3d31c154292574` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:20.680516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5138da-9014-45fd-b799-bc366f613c0c None admin] RBAC: Authorizing `identity:get_user(user_id=e8f8904939e24284be3d31c154292574)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:20.681939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5138da-9014-45fd-b799-bc366f613c0c None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:20.683617 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 564/1143] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:09:20 2022] GET /identity/v3/users/e8f8904939e24284be3d31c154292574 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:20.690315 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1216455-f640-4c11-bfe2-01b584544f39 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:20.747000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:20.787489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1216455-f640-4c11-bfe2-01b584544f39 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:20.819955 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d460ffe-3a42-4928-86f9-e78fda66bdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:20.820561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d460ffe-3a42-4928-86f9-e78fda66bdb2 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:20.820842 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d460ffe-3a42-4928-86f9-e78fda66bdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:20.821116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d460ffe-3a42-4928-86f9-e78fda66bdb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:20.887959 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d460ffe-3a42-4928-86f9-e78fda66bdb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:20.890082 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d460ffe-3a42-4928-86f9-e78fda66bdb2 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:20.943604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:20.989914 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 580/1144] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:09:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:09:20.995871 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5138da-9014-45fd-b799-bc366f613c0c None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:21.092325 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:21.095630 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5138da-9014-45fd-b799-bc366f613c0c None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:21.131488 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48843e5b-e961-41dd-9554-7a2f3d224dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:21.132114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48843e5b-e961-41dd-9554-7a2f3d224dfe None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:21.132393 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48843e5b-e961-41dd-9554-7a2f3d224dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:21.132665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48843e5b-e961-41dd-9554-7a2f3d224dfe None admin] PATH_INFO: `/v3/projects/2ababb1e974b4c7f964fcc86b4f67215` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:21.133774 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48843e5b-e961-41dd-9554-7a2f3d224dfe None admin] RBAC: Authorizing `identity:get_project(project_id=2ababb1e974b4c7f964fcc86b4f67215)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:21.135224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48843e5b-e961-41dd-9554-7a2f3d224dfe None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:21.136854 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 565/1145] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:09:20 2022] GET /identity/v3/projects/2ababb1e974b4c7f964fcc86b4f67215 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:21.143077 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d460ffe-3a42-4928-86f9-e78fda66bdb2 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:21.239187 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d460ffe-3a42-4928-86f9-e78fda66bdb2 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:21.272584 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595dec5-8bb2-45cf-abe0-cd067c45cb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:21.273179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595dec5-8bb2-45cf-abe0-cd067c45cb3e None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:21.273462 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595dec5-8bb2-45cf-abe0-cd067c45cb3e None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:21.273738 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595dec5-8bb2-45cf-abe0-cd067c45cb3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:21.274461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595dec5-8bb2-45cf-abe0-cd067c45cb3e None admin] RBAC: Adding query filter params (role.id=34df06ce84ad4cf8b3bd1a4490228017, scope.project.id=2ababb1e974b4c7f964fcc86b4f67215, user.id=e8f8904939e24284be3d31c154292574) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:09:21.274773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595dec5-8bb2-45cf-abe0-cd067c45cb3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:21.276220 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595dec5-8bb2-45cf-abe0-cd067c45cb3e None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:21.281692 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 581/1146] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:09:21 2022] GET /identity/v3/role_assignments?user.id=e8f8904939e24284be3d31c154292574&scope.project.id=2ababb1e974b4c7f964fcc86b4f67215&role.id=34df06ce84ad4cf8b3bd1a4490228017 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:23.166089 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786888d-40cb-43aa-9f04-eb76689ba17d None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:23.166669 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786888d-40cb-43aa-9f04-eb76689ba17d None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:23.166669 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786888d-40cb-43aa-9f04-eb76689ba17d None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:23.168002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 566/1147] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:09:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:09:23.174032 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c27846-2979-4e61-a70c-b72a45fdafb0 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:23.174412 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c27846-2979-4e61-a70c-b72a45fdafb0 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:23.174589 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c27846-2979-4e61-a70c-b72a45fdafb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:23.193164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-92c27846-2979-4e61-a70c-b72a45fdafb0 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:09:23.194348 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-92c27846-2979-4e61-a70c-b72a45fdafb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:09:23.211070 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:23.261508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 582/1148] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:09:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:09:23.532840 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786888d-40cb-43aa-9f04-eb76689ba17d None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:23.634222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786888d-40cb-43aa-9f04-eb76689ba17d None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:23.667261 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca817ae-ce70-42c5-a98c-e5dc4d82cc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:23.667879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca817ae-ce70-42c5-a98c-e5dc4d82cc2f None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:23.668156 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca817ae-ce70-42c5-a98c-e5dc4d82cc2f None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:23.668427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca817ae-ce70-42c5-a98c-e5dc4d82cc2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:23.678696 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca817ae-ce70-42c5-a98c-e5dc4d82cc2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:09:23.683427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-5ca817ae-ce70-42c5-a98c-e5dc4d82cc2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:09:23.683427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:09:23.683427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:09:23.683427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:09:23.683427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:09:23.683427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:09:23.683427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:09:23.683427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:09:23.683427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:09:23.683427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:09:23.683427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:09:23.683427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:09:23.683427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:09:23.683427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:09:23.683427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:09:23.683427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:09:23.684526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:09:23.684526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:09:23.684526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:09:23.684526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:09:23.684526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:09:23.684526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:09:23.684526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:09:23.684526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:09:23.684526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:09:23.684526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:09:23.684526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:09:23.684526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:09:23.684526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:09:23.684526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:09:23.684526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:09:23.684526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:09:23.685367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:09:23.685367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:09:23.685367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:09:23.685367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:09:23.685367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:09:23.685367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:09:23.685367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:09:23.685367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:09:23.685367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:09:23.685367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:09:23.685367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:09:23.685367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:09:23.685367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 567/1149] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:09:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:09:23.691293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c27846-2979-4e61-a70c-b72a45fdafb0 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:23.788294 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c27846-2979-4e61-a70c-b72a45fdafb0 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:23.824000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8c36c-ffc6-462e-993f-248dd67b7790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:23.824613 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8c36c-ffc6-462e-993f-248dd67b7790 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:23.824896 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8c36c-ffc6-462e-993f-248dd67b7790 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:23.825171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8c36c-ffc6-462e-993f-248dd67b7790 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:23.825831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8c36c-ffc6-462e-993f-248dd67b7790 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:09:23.826135 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8c36c-ffc6-462e-993f-248dd67b7790 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:23.827805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8c36c-ffc6-462e-993f-248dd67b7790 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:23.835585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 583/1150] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:09:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:23.841468 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca817ae-ce70-42c5-a98c-e5dc4d82cc2f None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:23.870526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:23.942307 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca817ae-ce70-42c5-a98c-e5dc4d82cc2f None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:23.975886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131d22b-837a-49d0-9d48-071406c80c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:23.976514 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131d22b-837a-49d0-9d48-071406c80c63 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:23.976793 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131d22b-837a-49d0-9d48-071406c80c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:23.977065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131d22b-837a-49d0-9d48-071406c80c63 None admin] PATH_INFO: `/v3/users/e8f8904939e24284be3d31c154292574` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:23.978187 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131d22b-837a-49d0-9d48-071406c80c63 None admin] RBAC: Authorizing `identity:get_user(user_id=e8f8904939e24284be3d31c154292574)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:23.979651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131d22b-837a-49d0-9d48-071406c80c63 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:23.981232 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 568/1151] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:09:23 2022] GET /identity/v3/users/e8f8904939e24284be3d31c154292574 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:09:23.987886 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8c36c-ffc6-462e-993f-248dd67b7790 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:24.004366 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:24.085667 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8c36c-ffc6-462e-993f-248dd67b7790 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:24.118319 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ebb425-ae14-4807-be81-b0b6d3c11730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:24.118948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebb425-ae14-4807-be81-b0b6d3c11730 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:24.119254 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebb425-ae14-4807-be81-b0b6d3c11730 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:24.119528 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebb425-ae14-4807-be81-b0b6d3c11730 None admin] PATH_INFO: `/v3/projects/2ababb1e974b4c7f964fcc86b4f67215` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:24.120630 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ebb425-ae14-4807-be81-b0b6d3c11730 None admin] RBAC: Authorizing `identity:get_project(project_id=2ababb1e974b4c7f964fcc86b4f67215)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:24.122054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ebb425-ae14-4807-be81-b0b6d3c11730 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:24.123837 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 584/1152] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:09:23 2022] GET /identity/v3/projects/2ababb1e974b4c7f964fcc86b4f67215 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:24.130082 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131d22b-837a-49d0-9d48-071406c80c63 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:24.222985 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:24.229086 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131d22b-837a-49d0-9d48-071406c80c63 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:24.262621 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa842e-5fdc-45d9-84d1-404e8e417aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:24.263293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa842e-5fdc-45d9-84d1-404e8e417aef None admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:24.263572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa842e-5fdc-45d9-84d1-404e8e417aef None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:24.263843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa842e-5fdc-45d9-84d1-404e8e417aef None admin] PATH_INFO: `/v3/projects/2ababb1e974b4c7f964fcc86b4f67215/users/e8f8904939e24284be3d31c154292574/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:24.265791 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa842e-5fdc-45d9-84d1-404e8e417aef None admin] RBAC: Authorizing `identity:create_grant(project_id=2ababb1e974b4c7f964fcc86b4f67215, user_id=e8f8904939e24284be3d31c154292574, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:24.267382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa842e-5fdc-45d9-84d1-404e8e417aef None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:24.280490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 569/1153] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 31 00:09:24 2022] PUT /identity/v3/projects/2ababb1e974b4c7f964fcc86b4f67215/users/e8f8904939e24284be3d31c154292574/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:09:26.145412 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f0122-ce5e-443d-8749-ab2aa04a389d None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:26.145928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f0122-ce5e-443d-8749-ab2aa04a389d None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:26.146041 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f0122-ce5e-443d-8749-ab2aa04a389d None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:26.147309 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 585/1154] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:09:26.152379 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e8583-4281-4ab6-8e6c-11be3e9d3d24 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:26.152794 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e8583-4281-4ab6-8e6c-11be3e9d3d24 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:26.152903 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e8583-4281-4ab6-8e6c-11be3e9d3d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:26.172151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-116e8583-4281-4ab6-8e6c-11be3e9d3d24 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:09:26.173309 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-116e8583-4281-4ab6-8e6c-11be3e9d3d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:09:26.238622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 570/1155] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:09:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:09:26.505376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f0122-ce5e-443d-8749-ab2aa04a389d None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:26.586683 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:26.605691 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f0122-ce5e-443d-8749-ab2aa04a389d None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:26.638545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c230e29-36bc-4d3c-b313-64bd9db6770e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:26.639730 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c230e29-36bc-4d3c-b313-64bd9db6770e None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:26.640006 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c230e29-36bc-4d3c-b313-64bd9db6770e None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:26.640272 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c230e29-36bc-4d3c-b313-64bd9db6770e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:26.650301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c230e29-36bc-4d3c-b313-64bd9db6770e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:09:26.655050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-8c230e29-36bc-4d3c-b313-64bd9db6770e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:09:26.655050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:09:26.655050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:09:26.655050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:09:26.655050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:09:26.655050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:09:26.655050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:09:26.655050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:09:26.655050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:09:26.655050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:09:26.655050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:09:26.655050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:09:26.655050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:09:26.655050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:09:26.655050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:09:26.655050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:09:26.656182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:09:26.656182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:09:26.656182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:09:26.656182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:09:26.656182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:09:26.656182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:09:26.656182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:09:26.656182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:09:26.656182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:09:26.656182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:09:26.656182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:09:26.656182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:09:26.656182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:09:26.656182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:09:26.656182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:09:26.656182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:09:26.657220 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:09:26.657220 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:09:26.657220 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:09:26.657220 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:09:26.657220 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:09:26.657220 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:09:26.657220 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:09:26.657220 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:09:26.657220 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:09:26.657220 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:09:26.657220 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:09:26.657220 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:09:26.657220 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 586/1156] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:09:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:09:26.662112 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e8583-4281-4ab6-8e6c-11be3e9d3d24 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:26.745144 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:26.761270 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e8583-4281-4ab6-8e6c-11be3e9d3d24 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:26.794427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8f197-931a-4293-8e8f-72479f2af754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:26.795066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8f197-931a-4293-8e8f-72479f2af754 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:26.795384 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8f197-931a-4293-8e8f-72479f2af754 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:26.795657 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8f197-931a-4293-8e8f-72479f2af754 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:26.796317 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8f197-931a-4293-8e8f-72479f2af754 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:09:26.796618 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8f197-931a-4293-8e8f-72479f2af754 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:26.797978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8f197-931a-4293-8e8f-72479f2af754 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:26.805783 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 571/1157] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:09:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:26.812560 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c230e29-36bc-4d3c-b313-64bd9db6770e None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:26.912159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c230e29-36bc-4d3c-b313-64bd9db6770e None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:26.944689 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e698b6-e9cc-4f8f-bf37-a8de95e19c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:26.945308 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e698b6-e9cc-4f8f-bf37-a8de95e19c88 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:26.945591 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e698b6-e9cc-4f8f-bf37-a8de95e19c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:26.945868 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e698b6-e9cc-4f8f-bf37-a8de95e19c88 None admin] PATH_INFO: `/v3/users/e8f8904939e24284be3d31c154292574` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:26.946986 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e698b6-e9cc-4f8f-bf37-a8de95e19c88 None admin] RBAC: Authorizing `identity:get_user(user_id=e8f8904939e24284be3d31c154292574)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:26.948447 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e698b6-e9cc-4f8f-bf37-a8de95e19c88 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:26.950024 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 587/1158] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:09:26 2022] GET /identity/v3/users/e8f8904939e24284be3d31c154292574 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:26.956818 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8f197-931a-4293-8e8f-72479f2af754 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:27.059227 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8f197-931a-4293-8e8f-72479f2af754 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:27.091847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e66e3-e532-4e04-83f2-cf95791e08bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:27.093052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e66e3-e532-4e04-83f2-cf95791e08bb None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:27.093328 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e66e3-e532-4e04-83f2-cf95791e08bb None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:27.093601 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e66e3-e532-4e04-83f2-cf95791e08bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:27.111775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:27.161047 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e66e3-e532-4e04-83f2-cf95791e08bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:27.162489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e66e3-e532-4e04-83f2-cf95791e08bb None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:27.261795 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 572/1159] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:09:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:09:27.268366 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e698b6-e9cc-4f8f-bf37-a8de95e19c88 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:27.281583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:27.366609 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e698b6-e9cc-4f8f-bf37-a8de95e19c88 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:27.399887 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b51bf2-8044-4ce0-a5f4-9969b6609417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:27.400550 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b51bf2-8044-4ce0-a5f4-9969b6609417 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:27.400828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b51bf2-8044-4ce0-a5f4-9969b6609417 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:27.401101 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b51bf2-8044-4ce0-a5f4-9969b6609417 None admin] PATH_INFO: `/v3/projects/2ababb1e974b4c7f964fcc86b4f67215` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:27.402213 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b51bf2-8044-4ce0-a5f4-9969b6609417 None admin] RBAC: Authorizing `identity:get_project(project_id=2ababb1e974b4c7f964fcc86b4f67215)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:27.403678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b51bf2-8044-4ce0-a5f4-9969b6609417 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:27.405327 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 588/1160] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:09:27 2022] GET /identity/v3/projects/2ababb1e974b4c7f964fcc86b4f67215 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:27.411480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e66e3-e532-4e04-83f2-cf95791e08bb None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:27.459230 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:27.508960 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e66e3-e532-4e04-83f2-cf95791e08bb None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:27.541868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2383e0-6d45-4697-b24b-f77b9933c084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:27.542462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2383e0-6d45-4697-b24b-f77b9933c084 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:27.542739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2383e0-6d45-4697-b24b-f77b9933c084 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:27.543010 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2383e0-6d45-4697-b24b-f77b9933c084 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:27.543757 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2383e0-6d45-4697-b24b-f77b9933c084 None admin] RBAC: Adding query filter params (role.id=34df06ce84ad4cf8b3bd1a4490228017, scope.project.id=2ababb1e974b4c7f964fcc86b4f67215, user.id=e8f8904939e24284be3d31c154292574) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:09:27.544068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2383e0-6d45-4697-b24b-f77b9933c084 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:27.545610 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2383e0-6d45-4697-b24b-f77b9933c084 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:27.551314 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 573/1161] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:09:27 2022] GET /identity/v3/role_assignments?user.id=e8f8904939e24284be3d31c154292574&scope.project.id=2ababb1e974b4c7f964fcc86b4f67215&role.id=34df06ce84ad4cf8b3bd1a4490228017 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:29.487145 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f9e24-4ed7-42d6-92c4-a61de1cb6c75 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:29.487676 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f9e24-4ed7-42d6-92c4-a61de1cb6c75 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:29.487676 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f9e24-4ed7-42d6-92c4-a61de1cb6c75 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:29.489042 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 589/1162] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:09:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:09:29.494267 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba98c9f-dcc6-4437-87b4-1ed48ecbbe0d None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:29.494695 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba98c9f-dcc6-4437-87b4-1ed48ecbbe0d None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:29.494821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba98c9f-dcc6-4437-87b4-1ed48ecbbe0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:29.513955 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-4ba98c9f-dcc6-4437-87b4-1ed48ecbbe0d None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:09:29.515424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-4ba98c9f-dcc6-4437-87b4-1ed48ecbbe0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:09:29.583794 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 574/1163] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:09:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:09:29.856336 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f9e24-4ed7-42d6-92c4-a61de1cb6c75 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:29.933788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:29.956830 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f9e24-4ed7-42d6-92c4-a61de1cb6c75 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:29.988959 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4feae36-d64f-4971-9745-336f7f0eca10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:29.989572 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4feae36-d64f-4971-9745-336f7f0eca10 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:29.989853 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4feae36-d64f-4971-9745-336f7f0eca10 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:29.990128 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4feae36-d64f-4971-9745-336f7f0eca10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:30.057131 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4feae36-d64f-4971-9745-336f7f0eca10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:30.058573 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4feae36-d64f-4971-9745-336f7f0eca10 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:30.134499 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:30.160871 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 590/1164] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:09:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:09:30.166898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba98c9f-dcc6-4437-87b4-1ed48ecbbe0d None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:30.209480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:30.267846 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba98c9f-dcc6-4437-87b4-1ed48ecbbe0d None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:30.301858 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81841f6-b026-47ce-9311-5bf73b88422c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:30.302483 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81841f6-b026-47ce-9311-5bf73b88422c None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:30.302765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81841f6-b026-47ce-9311-5bf73b88422c None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:30.303065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81841f6-b026-47ce-9311-5bf73b88422c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:30.309482 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81841f6-b026-47ce-9311-5bf73b88422c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:30.310917 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81841f6-b026-47ce-9311-5bf73b88422c None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:30.322444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-b81841f6-b026-47ce-9311-5bf73b88422c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 00:09:30.322444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:09:30.322444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 31 00:09:30.322444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 31 00:09:30.322444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:09:30.322444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:09:30.322444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:09:30.322444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:09:30.322444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 31 00:09:30.322444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:09:30.322444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:09:30.322444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:09:30.322444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:09:30.322444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:09:30.322444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:09:30.322444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:09:30.322444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:09:30.323823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:09:30.323823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:09:30.323823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:09:30.323823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:09:30.323823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:09:30.323823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:09:30.323823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:09:30.323823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:09:30.323823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:09:30.323823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:09:30.323823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:09:30.323823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:09:30.323823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:09:30.323823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:09:30.323823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:09:30.323823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:09:30.325070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:09:30.325070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:09:30.325070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:09:30.325070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:09:30.325070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:09:30.325070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:09:30.325070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:09:30.325070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:09:30.325070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:09:30.325070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:09:30.325070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 31 00:09:30.325070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:09:30.325070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 00:09:30.325070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:09:30.325070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 575/1165] 10.222.0.67 () {56 vars in 1117 bytes} [Wed Aug 31 00:09:30 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:09:30.330357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4feae36-d64f-4971-9745-336f7f0eca10 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:30.427695 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4feae36-d64f-4971-9745-336f7f0eca10 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:30.468391 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae23eec-ee19-42cb-a7b8-f34a4848b466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:30.469037 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae23eec-ee19-42cb-a7b8-f34a4848b466 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:30.469317 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae23eec-ee19-42cb-a7b8-f34a4848b466 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:30.469586 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae23eec-ee19-42cb-a7b8-f34a4848b466 None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:30.470224 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae23eec-ee19-42cb-a7b8-f34a4848b466 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:09:30.470526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae23eec-ee19-42cb-a7b8-f34a4848b466 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:30.471919 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae23eec-ee19-42cb-a7b8-f34a4848b466 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:30.478934 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 591/1166] 10.222.0.67 () {56 vars in 1115 bytes} [Wed Aug 31 00:09:30 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:32.400236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5986b7c-6a91-4e28-8ce6-87ec74ffb156 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:32.400751 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5986b7c-6a91-4e28-8ce6-87ec74ffb156 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:32.400856 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5986b7c-6a91-4e28-8ce6-87ec74ffb156 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:32.402136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 576/1167] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:09:32.407862 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a31ee8-5775-4c35-a990-6a481393e5d9 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:32.408281 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a31ee8-5775-4c35-a990-6a481393e5d9 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:32.408418 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a31ee8-5775-4c35-a990-6a481393e5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:32.427217 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-a3a31ee8-5775-4c35-a990-6a481393e5d9 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:09:32.428403 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a3a31ee8-5775-4c35-a990-6a481393e5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:09:32.463560 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:32.494685 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 592/1168] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:09:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:09:32.773338 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5986b7c-6a91-4e28-8ce6-87ec74ffb156 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:32.879092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5986b7c-6a91-4e28-8ce6-87ec74ffb156 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:32.880588 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:32.913591 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee0a0c-e0dc-4960-abf3-6e2919c14735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:32.914190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee0a0c-e0dc-4960-abf3-6e2919c14735 None admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:32.914470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee0a0c-e0dc-4960-abf3-6e2919c14735 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:32.914744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee0a0c-e0dc-4960-abf3-6e2919c14735 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:32.915444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee0a0c-e0dc-4960-abf3-6e2919c14735 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:32.916811 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee0a0c-e0dc-4960-abf3-6e2919c14735 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:32.932938 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 577/1169] 10.222.0.67 () {60 vars in 1138 bytes} [Wed Aug 31 00:09:32 2022] POST /identity/v3/domains => generated 267 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:09:34.876620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee24f0-f420-4987-a359-50a222b8d633 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:34.877180 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee24f0-f420-4987-a359-50a222b8d633 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:34.877180 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee24f0-f420-4987-a359-50a222b8d633 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:34.878488 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 593/1170] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:09:34.884223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5cfdc-e8af-42f6-802c-4c215e4caffe None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:34.884608 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5cfdc-e8af-42f6-802c-4c215e4caffe None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:34.884774 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5cfdc-e8af-42f6-802c-4c215e4caffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:34.902665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-daf5cfdc-e8af-42f6-802c-4c215e4caffe None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:09:34.903880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-daf5cfdc-e8af-42f6-802c-4c215e4caffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:09:34.969748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 578/1171] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:09:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:09:35.248548 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee24f0-f420-4987-a359-50a222b8d633 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:35.350461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee24f0-f420-4987-a359-50a222b8d633 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:35.385173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4f4cb-b496-4893-babb-e0352846df9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:35.385801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4f4cb-b496-4893-babb-e0352846df9b None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:35.386080 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4f4cb-b496-4893-babb-e0352846df9b None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:35.386353 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4f4cb-b496-4893-babb-e0352846df9b None admin] PATH_INFO: `/v3/domains/4786540725f149419586457c598b8f20` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:35.397320 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b4f4cb-b496-4893-babb-e0352846df9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=4786540725f149419586457c598b8f20)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:35.398757 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b4f4cb-b496-4893-babb-e0352846df9b None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:35.400371 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 594/1172] 10.222.0.67 () {56 vars in 1183 bytes} [Wed Aug 31 00:09:35 2022] GET /identity/v3/domains/4786540725f149419586457c598b8f20 => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:35.406969 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5cfdc-e8af-42f6-802c-4c215e4caffe None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:35.494688 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:35.504951 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5cfdc-e8af-42f6-802c-4c215e4caffe None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:35.537688 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa8c30-f2a5-42b0-9d2b-6230891538fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:35.538280 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa8c30-f2a5-42b0-9d2b-6230891538fb None admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:35.538555 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa8c30-f2a5-42b0-9d2b-6230891538fb None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:35.538821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa8c30-f2a5-42b0-9d2b-6230891538fb None admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:35.539496 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa8c30-f2a5-42b0-9d2b-6230891538fb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:35.540905 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa8c30-f2a5-42b0-9d2b-6230891538fb None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:35.566455 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 579/1173] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 31 00:09:35 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:09:37.523376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633432eb-c889-4697-9e51-1535543f3fa0 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:37.523941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633432eb-c889-4697-9e51-1535543f3fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:37.523941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633432eb-c889-4697-9e51-1535543f3fa0 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:37.525259 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 595/1174] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:09:37.531020 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b979d0-9fcc-4294-b2d9-9d8f5958ce4e None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:37.531546 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b979d0-9fcc-4294-b2d9-9d8f5958ce4e None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:37.531656 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b979d0-9fcc-4294-b2d9-9d8f5958ce4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:37.549995 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-b7b979d0-9fcc-4294-b2d9-9d8f5958ce4e None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:09:37.551212 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b7b979d0-9fcc-4294-b2d9-9d8f5958ce4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:09:37.616633 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 580/1175] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:09:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:09:37.886930 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633432eb-c889-4697-9e51-1535543f3fa0 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:37.917492 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:37.987340 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633432eb-c889-4697-9e51-1535543f3fa0 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:38.020190 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6144c-6b99-4908-9b05-73478d196e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:38.020816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6144c-6b99-4908-9b05-73478d196e0a None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:38.021092 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6144c-6b99-4908-9b05-73478d196e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:38.021361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6144c-6b99-4908-9b05-73478d196e0a None admin] PATH_INFO: `/v3/domains/4786540725f149419586457c598b8f20` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:38.022470 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6144c-6b99-4908-9b05-73478d196e0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=4786540725f149419586457c598b8f20)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:38.023929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6144c-6b99-4908-9b05-73478d196e0a None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:38.025520 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 596/1176] 10.222.0.67 () {56 vars in 1183 bytes} [Wed Aug 31 00:09:37 2022] GET /identity/v3/domains/4786540725f149419586457c598b8f20 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:38.031833 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b979d0-9fcc-4294-b2d9-9d8f5958ce4e None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:38.103050 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:38.128562 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b979d0-9fcc-4294-b2d9-9d8f5958ce4e None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:38.163990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e204827-d101-4c7f-99c2-7bda56df7c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:38.164529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e204827-d101-4c7f-99c2-7bda56df7c3f None admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:38.164808 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e204827-d101-4c7f-99c2-7bda56df7c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:38.165078 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e204827-d101-4c7f-99c2-7bda56df7c3f None admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:38.165718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e204827-d101-4c7f-99c2-7bda56df7c3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:38.167162 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e204827-d101-4c7f-99c2-7bda56df7c3f None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:38.188505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 581/1177] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 31 00:09:38 2022] POST /identity/v3/users => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:09:40.147716 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0b190-84c6-4707-bffd-bc2ef990b5df None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:40.148249 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0b190-84c6-4707-bffd-bc2ef990b5df None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:40.148249 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0b190-84c6-4707-bffd-bc2ef990b5df None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:40.149598 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 597/1178] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:09:40.154767 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75a99b-fc6b-48db-9276-b91dadc24603 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:40.155183 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75a99b-fc6b-48db-9276-b91dadc24603 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:40.155984 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75a99b-fc6b-48db-9276-b91dadc24603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:40.173650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-3c75a99b-fc6b-48db-9276-b91dadc24603 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:09:40.174815 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-3c75a99b-fc6b-48db-9276-b91dadc24603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:09:40.238265 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 582/1179] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:09:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:09:40.503862 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a0b190-84c6-4707-bffd-bc2ef990b5df None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:40.603468 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:40.607552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a0b190-84c6-4707-bffd-bc2ef990b5df None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:40.641784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55374b8d-b0f7-4976-81ca-1754271cd1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:40.642410 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55374b8d-b0f7-4976-81ca-1754271cd1e6 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:40.642692 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55374b8d-b0f7-4976-81ca-1754271cd1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:40.642965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55374b8d-b0f7-4976-81ca-1754271cd1e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:40.653156 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55374b8d-b0f7-4976-81ca-1754271cd1e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:09:40.658326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-55374b8d-b0f7-4976-81ca-1754271cd1e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:09:40.658326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:09:40.658326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:09:40.658326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:09:40.658326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:09:40.658326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:09:40.658326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:09:40.658326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:09:40.658326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:09:40.658326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:09:40.658326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:09:40.658326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:09:40.658326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:09:40.658326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:09:40.658326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:09:40.658326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:09:40.659535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:09:40.659535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:09:40.659535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:09:40.659535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:09:40.659535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:09:40.659535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:09:40.659535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:09:40.659535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:09:40.659535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:09:40.659535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:09:40.659535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:09:40.659535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:09:40.659535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:09:40.659535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:09:40.659535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:09:40.659535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:09:40.660572 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:09:40.660572 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:09:40.660572 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:09:40.660572 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:09:40.660572 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:09:40.660572 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:09:40.660572 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:09:40.660572 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:09:40.660572 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:09:40.660572 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:09:40.660572 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:09:40.660572 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:09:40.660572 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 598/1180] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:09:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:09:40.665452 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75a99b-fc6b-48db-9276-b91dadc24603 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:40.722276 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:40.767380 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75a99b-fc6b-48db-9276-b91dadc24603 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:40.800705 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0a5d2-dc53-4b3b-acd9-fb8740ed68dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:40.801308 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0a5d2-dc53-4b3b-acd9-fb8740ed68dd None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:40.801585 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0a5d2-dc53-4b3b-acd9-fb8740ed68dd None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:40.801854 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0a5d2-dc53-4b3b-acd9-fb8740ed68dd None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:40.802510 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0a5d2-dc53-4b3b-acd9-fb8740ed68dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:09:40.802812 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0a5d2-dc53-4b3b-acd9-fb8740ed68dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:40.804197 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0a5d2-dc53-4b3b-acd9-fb8740ed68dd None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:40.812649 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 583/1181] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:09:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:40.818623 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55374b8d-b0f7-4976-81ca-1754271cd1e6 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:40.913849 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55374b8d-b0f7-4976-81ca-1754271cd1e6 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:40.945717 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54941465-4216-4360-a24e-51a45c5458d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:40.946329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54941465-4216-4360-a24e-51a45c5458d8 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:40.946605 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54941465-4216-4360-a24e-51a45c5458d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:40.946872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54941465-4216-4360-a24e-51a45c5458d8 None admin] PATH_INFO: `/v3/users/02689819d45b4f6bb869b39e5ef023e6` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:40.961271 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54941465-4216-4360-a24e-51a45c5458d8 None admin] RBAC: Authorizing `identity:get_user(user_id=02689819d45b4f6bb869b39e5ef023e6)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:40.962691 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54941465-4216-4360-a24e-51a45c5458d8 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:40.964308 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 599/1182] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:09:40 2022] GET /identity/v3/users/02689819d45b4f6bb869b39e5ef023e6 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:40.971159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0a5d2-dc53-4b3b-acd9-fb8740ed68dd None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:41.070056 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0a5d2-dc53-4b3b-acd9-fb8740ed68dd None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:41.092405 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:41.103945 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7a7c9-e75b-4403-9187-928455eae6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:41.104547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7a7c9-e75b-4403-9187-928455eae6e8 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:41.104823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7a7c9-e75b-4403-9187-928455eae6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:41.105093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7a7c9-e75b-4403-9187-928455eae6e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:41.171147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7a7c9-e75b-4403-9187-928455eae6e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:41.172764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7a7c9-e75b-4403-9187-928455eae6e8 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:41.269327 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 584/1183] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:09:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:09:41.275354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54941465-4216-4360-a24e-51a45c5458d8 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:41.293730 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:41.371528 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54941465-4216-4360-a24e-51a45c5458d8 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:41.404514 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f80d21-83df-4402-ba1a-d761795f41a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:41.405125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f80d21-83df-4402-ba1a-d761795f41a2 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:41.406037 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f80d21-83df-4402-ba1a-d761795f41a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:41.406302 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f80d21-83df-4402-ba1a-d761795f41a2 None admin] PATH_INFO: `/v3/projects/4dc7b96438064a5794760cddb46fd128` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:41.407447 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f80d21-83df-4402-ba1a-d761795f41a2 None admin] RBAC: Authorizing `identity:get_project(project_id=4dc7b96438064a5794760cddb46fd128)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:41.408862 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f80d21-83df-4402-ba1a-d761795f41a2 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:41.410456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 600/1184] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:09:41 2022] GET /identity/v3/projects/4dc7b96438064a5794760cddb46fd128 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:41.416524 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7a7c9-e75b-4403-9187-928455eae6e8 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:41.440705 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:41.514242 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7a7c9-e75b-4403-9187-928455eae6e8 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:41.548621 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e907e9-0f94-4307-94c1-8896a34dfca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:41.549218 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e907e9-0f94-4307-94c1-8896a34dfca2 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:41.549496 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e907e9-0f94-4307-94c1-8896a34dfca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:41.549767 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e907e9-0f94-4307-94c1-8896a34dfca2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:41.550484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e907e9-0f94-4307-94c1-8896a34dfca2 None admin] RBAC: Adding query filter params (role.id=34df06ce84ad4cf8b3bd1a4490228017, scope.project.id=4dc7b96438064a5794760cddb46fd128, user.id=02689819d45b4f6bb869b39e5ef023e6) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:09:41.550790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e907e9-0f94-4307-94c1-8896a34dfca2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:41.552238 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e907e9-0f94-4307-94c1-8896a34dfca2 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:41.557473 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 585/1185] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:09:41 2022] GET /identity/v3/role_assignments?user.id=02689819d45b4f6bb869b39e5ef023e6&scope.project.id=4dc7b96438064a5794760cddb46fd128&role.id=34df06ce84ad4cf8b3bd1a4490228017 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:43.451855 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb63fa-206d-4545-9087-946154e84c7e None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:43.452452 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb63fa-206d-4545-9087-946154e84c7e None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:43.452720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb63fa-206d-4545-9087-946154e84c7e None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:43.454058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 601/1186] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:09:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:09:43.459154 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f133a-5253-4c7f-a754-b3c24d16de1c None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:43.459568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f133a-5253-4c7f-a754-b3c24d16de1c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:43.459708 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f133a-5253-4c7f-a754-b3c24d16de1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:43.478262 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-f86f133a-5253-4c7f-a754-b3c24d16de1c None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:09:43.479476 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f86f133a-5253-4c7f-a754-b3c24d16de1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:09:43.543941 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 586/1187] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:09:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:09:43.810929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb63fa-206d-4545-9087-946154e84c7e None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:43.894218 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:43.910153 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb63fa-206d-4545-9087-946154e84c7e None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:43.942341 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaee1af-c717-4719-9034-226a5c9c8e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:43.942952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaee1af-c717-4719-9034-226a5c9c8e4e None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:43.943251 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaee1af-c717-4719-9034-226a5c9c8e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:43.943516 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaee1af-c717-4719-9034-226a5c9c8e4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:43.953930 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaee1af-c717-4719-9034-226a5c9c8e4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:09:43.958563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-5aaee1af-c717-4719-9034-226a5c9c8e4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:09:43.958563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:09:43.958563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:09:43.958563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:09:43.958563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:09:43.958563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:09:43.958563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:09:43.958563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:09:43.958563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:09:43.958563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:09:43.958563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:09:43.958563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:09:43.958563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:09:43.958563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:09:43.958563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:09:43.958563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:09:43.959674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:09:43.959674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:09:43.959674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:09:43.959674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:09:43.959674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:09:43.959674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:09:43.959674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:09:43.959674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:09:43.959674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:09:43.959674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:09:43.959674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:09:43.959674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:09:43.959674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:09:43.959674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:09:43.959674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:09:43.959674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:09:43.960946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:09:43.960946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:09:43.960946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:09:43.960946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:09:43.960946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:09:43.960946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:09:43.960946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:09:43.960946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:09:43.960946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:09:43.960946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:09:43.960946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:09:43.960946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:09:43.960946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 602/1188] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:09:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:09:43.965683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f133a-5253-4c7f-a754-b3c24d16de1c None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:43.983215 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:44.063921 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f133a-5253-4c7f-a754-b3c24d16de1c None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:44.096139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b4769-d218-48a8-bd37-eefab730802a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:44.096741 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b4769-d218-48a8-bd37-eefab730802a None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:44.097018 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b4769-d218-48a8-bd37-eefab730802a None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:44.097286 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b4769-d218-48a8-bd37-eefab730802a None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:44.097932 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b4769-d218-48a8-bd37-eefab730802a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:09:44.098231 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b4769-d218-48a8-bd37-eefab730802a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:44.099608 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b4769-d218-48a8-bd37-eefab730802a None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:44.107219 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 587/1189] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:09:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:44.113876 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaee1af-c717-4719-9034-226a5c9c8e4e None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:44.210877 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaee1af-c717-4719-9034-226a5c9c8e4e None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:44.244597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c4f42-f1f9-46e0-b4c1-2569201acf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:44.245216 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c4f42-f1f9-46e0-b4c1-2569201acf27 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:44.245497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c4f42-f1f9-46e0-b4c1-2569201acf27 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:44.245772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c4f42-f1f9-46e0-b4c1-2569201acf27 None admin] PATH_INFO: `/v3/users/02689819d45b4f6bb869b39e5ef023e6` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:44.246886 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c4f42-f1f9-46e0-b4c1-2569201acf27 None admin] RBAC: Authorizing `identity:get_user(user_id=02689819d45b4f6bb869b39e5ef023e6)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:44.248344 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c4f42-f1f9-46e0-b4c1-2569201acf27 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:44.250546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 603/1190] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:09:44 2022] GET /identity/v3/users/02689819d45b4f6bb869b39e5ef023e6 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:44.256599 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b4769-d218-48a8-bd37-eefab730802a None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:44.333344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:44.354838 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b4769-d218-48a8-bd37-eefab730802a None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:44.389244 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8e2f1-ddac-4f2d-9727-c2ab00fbad8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:44.389918 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8e2f1-ddac-4f2d-9727-c2ab00fbad8c None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:44.390191 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8e2f1-ddac-4f2d-9727-c2ab00fbad8c None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:44.390463 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8e2f1-ddac-4f2d-9727-c2ab00fbad8c None admin] PATH_INFO: `/v3/projects/4dc7b96438064a5794760cddb46fd128` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:44.391596 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e8e2f1-ddac-4f2d-9727-c2ab00fbad8c None admin] RBAC: Authorizing `identity:get_project(project_id=4dc7b96438064a5794760cddb46fd128)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:44.393021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e8e2f1-ddac-4f2d-9727-c2ab00fbad8c None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:44.394653 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 588/1191] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:09:44 2022] GET /identity/v3/projects/4dc7b96438064a5794760cddb46fd128 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:44.400832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c4f42-f1f9-46e0-b4c1-2569201acf27 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:44.416812 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:44.502843 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c4f42-f1f9-46e0-b4c1-2569201acf27 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:44.535248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d838503-7d1a-46a8-b5c2-33b1e10e3991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:44.535894 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d838503-7d1a-46a8-b5c2-33b1e10e3991 None admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:44.536170 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d838503-7d1a-46a8-b5c2-33b1e10e3991 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:44.536437 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d838503-7d1a-46a8-b5c2-33b1e10e3991 None admin] PATH_INFO: `/v3/projects/4dc7b96438064a5794760cddb46fd128/users/02689819d45b4f6bb869b39e5ef023e6/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:44.538301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d838503-7d1a-46a8-b5c2-33b1e10e3991 None admin] RBAC: Authorizing `identity:create_grant(project_id=4dc7b96438064a5794760cddb46fd128, user_id=02689819d45b4f6bb869b39e5ef023e6, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:44.539752 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d838503-7d1a-46a8-b5c2-33b1e10e3991 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:44.552420 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 604/1192] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 31 00:09:44 2022] PUT /identity/v3/projects/4dc7b96438064a5794760cddb46fd128/users/02689819d45b4f6bb869b39e5ef023e6/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:09:46.522804 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504194b-b9a5-472a-b8c5-275949a6db0c None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:46.523384 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504194b-b9a5-472a-b8c5-275949a6db0c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:46.523384 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504194b-b9a5-472a-b8c5-275949a6db0c None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:46.524709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 589/1193] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:09:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:09:46.529919 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0baf3e7-aec4-483a-b496-ee83c4177a18 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:46.530310 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0baf3e7-aec4-483a-b496-ee83c4177a18 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:46.531007 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0baf3e7-aec4-483a-b496-ee83c4177a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:46.556635 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-a0baf3e7-aec4-483a-b496-ee83c4177a18 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:09:46.557820 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a0baf3e7-aec4-483a-b496-ee83c4177a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:09:46.630432 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 605/1194] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:09:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:09:46.902956 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504194b-b9a5-472a-b8c5-275949a6db0c None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:47.020176 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504194b-b9a5-472a-b8c5-275949a6db0c None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:47.052806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d5e8d-9d15-4f3c-9370-0dfc06782099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:47.053415 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d5e8d-9d15-4f3c-9370-0dfc06782099 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:47.053690 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d5e8d-9d15-4f3c-9370-0dfc06782099 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:47.053957 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d5e8d-9d15-4f3c-9370-0dfc06782099 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:47.064714 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4d5e8d-9d15-4f3c-9370-0dfc06782099 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:09:47.069414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-2b4d5e8d-9d15-4f3c-9370-0dfc06782099 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:09:47.069414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:09:47.069414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:09:47.069414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:09:47.069414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:09:47.069414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:09:47.069414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:09:47.069414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:09:47.069414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:09:47.069414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:09:47.069414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:09:47.069414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:09:47.069414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:09:47.069414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:09:47.069414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:09:47.069414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:09:47.070517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:09:47.070517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:09:47.070517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:09:47.070517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:09:47.070517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:09:47.070517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:09:47.070517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:09:47.070517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:09:47.070517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:09:47.070517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:09:47.070517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:09:47.070517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:09:47.070517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:09:47.070517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:09:47.070517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:09:47.070517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:09:47.071433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:09:47.071433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:09:47.071433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:09:47.071433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:09:47.071433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:09:47.071433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:09:47.071433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:09:47.071433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:09:47.071433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:09:47.071433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:09:47.071433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:09:47.071433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:09:47.071433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 590/1195] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:09:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:09:47.077032 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0baf3e7-aec4-483a-b496-ee83c4177a18 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:47.081081 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:47.179801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0baf3e7-aec4-483a-b496-ee83c4177a18 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:47.212571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427895c-5882-42dd-a5be-c5ddf64f86e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:47.213167 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427895c-5882-42dd-a5be-c5ddf64f86e3 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:47.213453 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427895c-5882-42dd-a5be-c5ddf64f86e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:47.213728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427895c-5882-42dd-a5be-c5ddf64f86e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:47.214386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427895c-5882-42dd-a5be-c5ddf64f86e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:09:47.214690 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427895c-5882-42dd-a5be-c5ddf64f86e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:47.216083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427895c-5882-42dd-a5be-c5ddf64f86e3 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:47.224627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 606/1196] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:09:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:47.230540 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d5e8d-9d15-4f3c-9370-0dfc06782099 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:47.240446 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:47.335601 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d5e8d-9d15-4f3c-9370-0dfc06782099 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:47.367864 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c149ca-73b7-477e-b473-4229ba0ce11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:47.368483 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c149ca-73b7-477e-b473-4229ba0ce11f None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:47.368760 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c149ca-73b7-477e-b473-4229ba0ce11f None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:47.369030 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c149ca-73b7-477e-b473-4229ba0ce11f None admin] PATH_INFO: `/v3/users/02689819d45b4f6bb869b39e5ef023e6` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:47.370128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c149ca-73b7-477e-b473-4229ba0ce11f None admin] RBAC: Authorizing `identity:get_user(user_id=02689819d45b4f6bb869b39e5ef023e6)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:47.371581 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c149ca-73b7-477e-b473-4229ba0ce11f None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:47.373154 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 591/1197] 10.222.0.67 () {56 vars in 1177 bytes} [Wed Aug 31 00:09:47 2022] GET /identity/v3/users/02689819d45b4f6bb869b39e5ef023e6 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:47.380310 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427895c-5882-42dd-a5be-c5ddf64f86e3 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:47.443369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:47.476833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427895c-5882-42dd-a5be-c5ddf64f86e3 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:47.510067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631dcdb-9923-4dd4-a515-61cbc8a0676d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:47.510665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631dcdb-9923-4dd4-a515-61cbc8a0676d None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:47.510944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631dcdb-9923-4dd4-a515-61cbc8a0676d None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:47.511241 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631dcdb-9923-4dd4-a515-61cbc8a0676d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:47.583660 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2631dcdb-9923-4dd4-a515-61cbc8a0676d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:47.585109 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2631dcdb-9923-4dd4-a515-61cbc8a0676d None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:47.646438 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:47.681203 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 607/1198] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:09:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:09:47.687183 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c149ca-73b7-477e-b473-4229ba0ce11f None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:47.759660 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:47.784088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c149ca-73b7-477e-b473-4229ba0ce11f None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:47.816517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293582c-5854-430e-b8e8-4f8afea41b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:47.817132 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293582c-5854-430e-b8e8-4f8afea41b89 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:47.817420 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293582c-5854-430e-b8e8-4f8afea41b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:47.817685 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293582c-5854-430e-b8e8-4f8afea41b89 None admin] PATH_INFO: `/v3/projects/4dc7b96438064a5794760cddb46fd128` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:47.818793 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c293582c-5854-430e-b8e8-4f8afea41b89 None admin] RBAC: Authorizing `identity:get_project(project_id=4dc7b96438064a5794760cddb46fd128)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:47.820251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c293582c-5854-430e-b8e8-4f8afea41b89 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:47.821859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 592/1199] 10.222.0.67 () {56 vars in 1186 bytes} [Wed Aug 31 00:09:47 2022] GET /identity/v3/projects/4dc7b96438064a5794760cddb46fd128 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:47.828635 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631dcdb-9923-4dd4-a515-61cbc8a0676d None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:09:47.923858 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631dcdb-9923-4dd4-a515-61cbc8a0676d None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:09:47.956846 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fea0f4-9a0f-4929-9579-f467566c2a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:09:47.957444 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fea0f4-9a0f-4929-9579-f467566c2a04 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:09:47.957725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fea0f4-9a0f-4929-9579-f467566c2a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:09:47.957998 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fea0f4-9a0f-4929-9579-f467566c2a04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:09:47.958721 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fea0f4-9a0f-4929-9579-f467566c2a04 None admin] RBAC: Adding query filter params (role.id=34df06ce84ad4cf8b3bd1a4490228017, scope.project.id=4dc7b96438064a5794760cddb46fd128, user.id=02689819d45b4f6bb869b39e5ef023e6) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:09:47.959030 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fea0f4-9a0f-4929-9579-f467566c2a04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:09:47.960486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fea0f4-9a0f-4929-9579-f467566c2a04 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:09:47.966055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 608/1200] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:09:47 2022] GET /identity/v3/role_assignments?user.id=02689819d45b4f6bb869b39e5ef023e6&scope.project.id=4dc7b96438064a5794760cddb46fd128&role.id=34df06ce84ad4cf8b3bd1a4490228017 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:09:57.811892 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=29 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:09:57.963505 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=85 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:10:45.228355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb642e-0e85-4800-8645-034a3488c977 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:45.229000 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb642e-0e85-4800-8645-034a3488c977 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:45.229000 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb642e-0e85-4800-8645-034a3488c977 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:45.230873 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 593/1201] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:10:45.236445 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687d2e3-a023-4bd9-b26c-0325dc5dffda None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:45.237546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687d2e3-a023-4bd9-b26c-0325dc5dffda None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:45.237546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687d2e3-a023-4bd9-b26c-0325dc5dffda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:45.267592 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-a687d2e3-a023-4bd9-b26c-0325dc5dffda None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:10:45.272222 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a687d2e3-a023-4bd9-b26c-0325dc5dffda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:10:45.423515 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 609/1202] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:10:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:10:45.719581 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb642e-0e85-4800-8645-034a3488c977 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:45.884776 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb642e-0e85-4800-8645-034a3488c977 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:45.933054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0b90b-a286-423c-a245-09d2c69758f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:45.933829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0b90b-a286-423c-a245-09d2c69758f1 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:45.934182 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0b90b-a286-423c-a245-09d2c69758f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:45.934458 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0b90b-a286-423c-a245-09d2c69758f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:45.950568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b0b90b-a286-423c-a245-09d2c69758f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:45.952538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b0b90b-a286-423c-a245-09d2c69758f1 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:45.977140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-92b0b90b-a286-423c-a245-09d2c69758f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:10:45.977140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:10:45.977140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:10:45.977140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:10:45.977140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:10:45.977140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:10:45.977140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:10:45.977140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:10:45.977140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:10:45.977140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:10:45.977140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:10:45.977140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:10:45.977140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:10:45.977140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:10:45.977140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:10:45.977140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:10:45.978241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:10:45.978241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:10:45.978241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:10:45.978241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:10:45.978241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:10:45.978241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:10:45.978241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:10:45.978241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:10:45.978241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:10:45.978241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:10:45.978241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:10:45.978241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:10:45.978241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:10:45.978241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:10:45.978241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:10:45.978241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:10:45.978241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:10:45.979279 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:10:45.979279 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:10:45.979279 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:10:45.979279 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 594/1203] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:10:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:10:45.982457 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687d2e3-a023-4bd9-b26c-0325dc5dffda None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:46.126938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687d2e3-a023-4bd9-b26c-0325dc5dffda None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:46.169804 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:10:46.179117 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647e18c-6c3a-423d-a8b4-e7f324b77cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:46.179826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647e18c-6c3a-423d-a8b4-e7f324b77cbb None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:46.180186 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647e18c-6c3a-423d-a8b4-e7f324b77cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:46.180666 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647e18c-6c3a-423d-a8b4-e7f324b77cbb None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:46.181665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1647e18c-6c3a-423d-a8b4-e7f324b77cbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:10:46.182085 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1647e18c-6c3a-423d-a8b4-e7f324b77cbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:46.183896 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1647e18c-6c3a-423d-a8b4-e7f324b77cbb None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:46.210385 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 610/1204] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:10:46.213685 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0b90b-a286-423c-a245-09d2c69758f1 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:46.292999 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:10:46.347456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0b90b-a286-423c-a245-09d2c69758f1 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:46.387468 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab52418-4564-44a6-8b0b-9095917ff864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:46.388087 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab52418-4564-44a6-8b0b-9095917ff864 None admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:46.388367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab52418-4564-44a6-8b0b-9095917ff864 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:46.388659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab52418-4564-44a6-8b0b-9095917ff864 None admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:46.389406 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab52418-4564-44a6-8b0b-9095917ff864 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:46.390908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab52418-4564-44a6-8b0b-9095917ff864 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:46.427085 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 595/1205] 10.222.0.67 () {60 vars in 1132 bytes} [Wed Aug 31 00:10:46 2022] POST /identity/v3/users => generated 251 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:10:48.789590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa26b99-652c-4693-ace3-5922c3dbb2b7 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:48.789590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa26b99-652c-4693-ace3-5922c3dbb2b7 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:48.789590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa26b99-652c-4693-ace3-5922c3dbb2b7 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:48.789590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 611/1206] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:10:48.795622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4041119-667c-47c6-823e-9d0d8230ff92 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:48.796032 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4041119-667c-47c6-823e-9d0d8230ff92 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:48.796180 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4041119-667c-47c6-823e-9d0d8230ff92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:48.816999 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-b4041119-667c-47c6-823e-9d0d8230ff92 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:10:48.818213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b4041119-667c-47c6-823e-9d0d8230ff92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:10:48.907917 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 596/1207] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:10:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:10:49.187692 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa26b99-652c-4693-ace3-5922c3dbb2b7 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:49.315954 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa26b99-652c-4693-ace3-5922c3dbb2b7 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:49.357546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7fa4f8-fd03-4c7b-881c-d34188fc613a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:49.358335 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fa4f8-fd03-4c7b-881c-d34188fc613a None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:49.358641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fa4f8-fd03-4c7b-881c-d34188fc613a None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:49.358973 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fa4f8-fd03-4c7b-881c-d34188fc613a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:49.373367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7fa4f8-fd03-4c7b-881c-d34188fc613a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:10:49.378116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-dc7fa4f8-fd03-4c7b-881c-d34188fc613a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:10:49.378116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:10:49.378116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:10:49.378116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:10:49.378116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:10:49.378116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:10:49.378116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:10:49.378116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:10:49.378116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:10:49.378116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:10:49.378116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:10:49.378116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:10:49.378116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:10:49.378116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:10:49.378116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:10:49.378116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:10:49.379164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:10:49.379164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:10:49.379164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:10:49.379164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:10:49.379164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:10:49.379164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:10:49.379164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:10:49.379164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:10:49.379164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:10:49.379164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:10:49.379164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:10:49.379164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:10:49.379164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:10:49.379164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:10:49.379164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:10:49.379164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:10:49.380054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:10:49.380054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:10:49.380054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:10:49.380054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:10:49.380054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:10:49.380054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:10:49.380054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:10:49.380054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:10:49.380054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:10:49.380054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:10:49.380054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:10:49.380054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:10:49.380054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 612/1208] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 31 00:10:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:10:49.385569 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4041119-667c-47c6-823e-9d0d8230ff92 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:49.432064 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:10:49.500490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4041119-667c-47c6-823e-9d0d8230ff92 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:49.538532 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10f530-7ab3-4c55-bcbc-7ec17186a0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:49.540104 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10f530-7ab3-4c55-bcbc-7ec17186a0ab None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:49.540104 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10f530-7ab3-4c55-bcbc-7ec17186a0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:49.540443 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10f530-7ab3-4c55-bcbc-7ec17186a0ab None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:49.541268 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb10f530-7ab3-4c55-bcbc-7ec17186a0ab None admin] RBAC: Adding query filter params (name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:10:49.541612 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb10f530-7ab3-4c55-bcbc-7ec17186a0ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:49.543204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb10f530-7ab3-4c55-bcbc-7ec17186a0ab None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:49.552723 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 597/1209] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 31 00:10:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:10:49.558810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7fa4f8-fd03-4c7b-881c-d34188fc613a None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:49.621810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:10:49.676106 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7fa4f8-fd03-4c7b-881c-d34188fc613a None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:49.717391 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5e0fc-31c6-4e08-8612-465c7e6de4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:49.718175 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5e0fc-31c6-4e08-8612-465c7e6de4db None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:49.718514 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5e0fc-31c6-4e08-8612-465c7e6de4db None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:49.718825 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5e0fc-31c6-4e08-8612-465c7e6de4db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:49.733995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee5e0fc-31c6-4e08-8612-465c7e6de4db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:49.733995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee5e0fc-31c6-4e08-8612-465c7e6de4db None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:49.749583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-aee5e0fc-31c6-4e08-8612-465c7e6de4db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:10:49.749583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:10:49.749583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:10:49.749583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:10:49.749583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:10:49.749583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:10:49.749583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:10:49.749583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:10:49.749583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:10:49.749583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:10:49.749583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:10:49.749583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:10:49.749583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:10:49.749583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:10:49.749583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:10:49.749583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:10:49.750826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:10:49.750826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:10:49.750826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:10:49.750826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:10:49.750826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:10:49.750826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:10:49.750826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:10:49.750826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:10:49.750826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:10:49.750826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:10:49.750826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:10:49.750826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:10:49.750826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:10:49.750826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:10:49.750826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:10:49.750826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:10:49.750826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:10:49.752292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:10:49.752292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:10:49.752292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:10:49.758126 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10f530-7ab3-4c55-bcbc-7ec17186a0ab None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:49.758838 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 613/1210] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:10:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:10:49.866272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:10:49.869544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10f530-7ab3-4c55-bcbc-7ec17186a0ab None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:49.910265 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a5f14-a22b-4d16-b4aa-c2ee50f993a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:49.910923 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a5f14-a22b-4d16-b4aa-c2ee50f993a2 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:49.911321 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a5f14-a22b-4d16-b4aa-c2ee50f993a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:49.911594 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a5f14-a22b-4d16-b4aa-c2ee50f993a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:49.912258 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503a5f14-a22b-4d16-b4aa-c2ee50f993a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:10:49.912542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503a5f14-a22b-4d16-b4aa-c2ee50f993a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:49.914351 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503a5f14-a22b-4d16-b4aa-c2ee50f993a2 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:49.928904 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 598/1211] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:10:49.934430 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5e0fc-31c6-4e08-8612-465c7e6de4db None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:50.059332 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5e0fc-31c6-4e08-8612-465c7e6de4db None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:50.064550 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:10:50.102460 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e465e58-0e7f-4ed8-9ef2-5124e09640bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:50.103152 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e465e58-0e7f-4ed8-9ef2-5124e09640bf None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:50.103448 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e465e58-0e7f-4ed8-9ef2-5124e09640bf None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:50.103810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e465e58-0e7f-4ed8-9ef2-5124e09640bf None admin] PATH_INFO: `/v3/users/placement` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:50.113549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e465e58-0e7f-4ed8-9ef2-5124e09640bf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:50.115388 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e465e58-0e7f-4ed8-9ef2-5124e09640bf None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:50.128795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-9e465e58-0e7f-4ed8-9ef2-5124e09640bf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 00:10:50.128795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:10:50.128795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:10:50.128795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:10:50.128795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:10:50.128795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:10:50.128795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:10:50.128795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:10:50.128795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:10:50.128795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:10:50.128795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:10:50.128795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:10:50.128795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:10:50.128795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:10:50.128795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:10:50.128795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:10:50.130034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:10:50.130034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:10:50.130034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:10:50.130034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:10:50.130034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:10:50.130034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:10:50.130034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:10:50.130034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:10:50.130034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:10:50.130034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:10:50.130034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:10:50.130034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:10:50.130034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:10:50.130034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:10:50.130034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:10:50.130034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:10:50.130034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:10:50.130034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:10:50.135957 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:10:50.135957 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:10:50.135957 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:10:50.135957 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:10:50.135957 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:10:50.135957 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:10:50.135957 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:10:50.135957 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:10:50.135957 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 00:10:50.135957 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:10:50.135957 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 614/1212] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:10:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:10:50.136583 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a5f14-a22b-4d16-b4aa-c2ee50f993a2 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:50.244543 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a5f14-a22b-4d16-b4aa-c2ee50f993a2 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:50.282498 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b854fb0-7e34-4256-8309-77917d3b553d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:50.283147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b854fb0-7e34-4256-8309-77917d3b553d None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:50.283424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b854fb0-7e34-4256-8309-77917d3b553d None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:50.283695 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b854fb0-7e34-4256-8309-77917d3b553d None admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:50.284735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b854fb0-7e34-4256-8309-77917d3b553d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:10:50.285043 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b854fb0-7e34-4256-8309-77917d3b553d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:50.286481 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b854fb0-7e34-4256-8309-77917d3b553d None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:50.304579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 599/1213] 10.222.0.67 () {56 vars in 1143 bytes} [Wed Aug 31 00:10:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:10:50.310529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e465e58-0e7f-4ed8-9ef2-5124e09640bf None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:50.429067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e465e58-0e7f-4ed8-9ef2-5124e09640bf None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:50.468338 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082671e7-ef5c-4dd1-8e32-a0f94582910b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:50.468989 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082671e7-ef5c-4dd1-8e32-a0f94582910b None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:50.469270 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082671e7-ef5c-4dd1-8e32-a0f94582910b None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:50.469544 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082671e7-ef5c-4dd1-8e32-a0f94582910b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:50.506820 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:10:50.552068 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082671e7-ef5c-4dd1-8e32-a0f94582910b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:50.555908 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082671e7-ef5c-4dd1-8e32-a0f94582910b None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:50.662615 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 615/1214] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:10:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:10:50.668224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b854fb0-7e34-4256-8309-77917d3b553d None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:50.688595 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:10:50.777537 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b854fb0-7e34-4256-8309-77917d3b553d None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:50.814565 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef96795-22aa-4d0f-a2cc-6a9deefe59cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:50.815274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef96795-22aa-4d0f-a2cc-6a9deefe59cf None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:50.815558 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef96795-22aa-4d0f-a2cc-6a9deefe59cf None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:50.815874 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef96795-22aa-4d0f-a2cc-6a9deefe59cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:50.826482 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef96795-22aa-4d0f-a2cc-6a9deefe59cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:50.827972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef96795-22aa-4d0f-a2cc-6a9deefe59cf None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:50.844879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-7ef96795-22aa-4d0f-a2cc-6a9deefe59cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:10:50.844879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:10:50.844879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:10:50.844879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:10:50.844879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:10:50.844879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:10:50.844879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:10:50.844879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:10:50.844879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:10:50.844879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:10:50.844879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:10:50.844879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:10:50.844879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:10:50.844879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:10:50.844879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:10:50.844879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:10:50.845991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:10:50.845991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:10:50.845991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:10:50.845991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:10:50.845991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:10:50.845991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:10:50.845991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:10:50.845991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:10:50.845991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:10:50.845991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:10:50.845991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:10:50.845991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:10:50.845991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:10:50.845991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:10:50.845991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:10:50.845991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:10:50.845991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:10:50.847664 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:10:50.847664 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:10:50.847664 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:10:50.847664 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 600/1215] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:10:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:10:50.853814 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082671e7-ef5c-4dd1-8e32-a0f94582910b None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:50.923558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:10:50.971230 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082671e7-ef5c-4dd1-8e32-a0f94582910b None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:51.011892 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7eb21-df75-4957-a070-08306510ebca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:51.012762 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7eb21-df75-4957-a070-08306510ebca None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:51.013140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7eb21-df75-4957-a070-08306510ebca None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:51.013535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7eb21-df75-4957-a070-08306510ebca None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:51.014461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd7eb21-df75-4957-a070-08306510ebca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:10:51.014877 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd7eb21-df75-4957-a070-08306510ebca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:51.016574 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd7eb21-df75-4957-a070-08306510ebca None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:51.031203 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 616/1216] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:10:51.036134 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef96795-22aa-4d0f-a2cc-6a9deefe59cf None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:51.124686 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:10:51.162422 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef96795-22aa-4d0f-a2cc-6a9deefe59cf None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:51.202567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa9035-1d6b-4e56-9a0e-cb85f19d3191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:51.203153 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa9035-1d6b-4e56-9a0e-cb85f19d3191 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:51.203430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa9035-1d6b-4e56-9a0e-cb85f19d3191 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:51.203697 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa9035-1d6b-4e56-9a0e-cb85f19d3191 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:51.212053 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa9035-1d6b-4e56-9a0e-cb85f19d3191 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:51.213668 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa9035-1d6b-4e56-9a0e-cb85f19d3191 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:51.226860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-d8fa9035-1d6b-4e56-9a0e-cb85f19d3191 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:10:51.226860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:10:51.226860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:10:51.226860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:10:51.226860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:10:51.226860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:10:51.226860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:10:51.226860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:10:51.226860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:10:51.226860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:10:51.226860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:10:51.226860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:10:51.226860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:10:51.226860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:10:51.226860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:10:51.228065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:10:51.228065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:10:51.228065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:10:51.228065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:10:51.228065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:10:51.228065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:10:51.228065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:10:51.228065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:10:51.228065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:10:51.228065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:10:51.228065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:10:51.228065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:10:51.228065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:10:51.228065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:10:51.228065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:10:51.228065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:10:51.228065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:10:51.228990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:10:51.228990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:10:51.228990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:10:51.228990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:10:51.228990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:10:51.228990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:10:51.228990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:10:51.228990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:10:51.228990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 601/1217] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:10:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:10:51.236403 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7eb21-df75-4957-a070-08306510ebca None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:51.377026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7eb21-df75-4957-a070-08306510ebca None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:51.406292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:10:51.421952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709f71c-a7a2-4f69-9b43-47bbca29ed88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:51.422314 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709f71c-a7a2-4f69-9b43-47bbca29ed88 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:51.422719 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709f71c-a7a2-4f69-9b43-47bbca29ed88 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:51.423339 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709f71c-a7a2-4f69-9b43-47bbca29ed88 None admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:51.424167 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709f71c-a7a2-4f69-9b43-47bbca29ed88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:10:51.425234 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709f71c-a7a2-4f69-9b43-47bbca29ed88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:51.427634 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d709f71c-a7a2-4f69-9b43-47bbca29ed88 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:51.449470 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 617/1218] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:10:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:10:51.450756 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa9035-1d6b-4e56-9a0e-cb85f19d3191 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:51.562251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa9035-1d6b-4e56-9a0e-cb85f19d3191 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:51.591543 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:10:51.600297 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72eecf-78dc-48f7-8048-98129a8c601b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:51.600906 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72eecf-78dc-48f7-8048-98129a8c601b None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:51.601186 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72eecf-78dc-48f7-8048-98129a8c601b None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:51.601464 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72eecf-78dc-48f7-8048-98129a8c601b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:51.602198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c72eecf-78dc-48f7-8048-98129a8c601b None admin] RBAC: Adding query filter params (role.id=4680aa64da80470b87075090fca2354b, scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb, user.id=fdacdcb2451742938164fdd44cdbaebe) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:10:51.602508 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c72eecf-78dc-48f7-8048-98129a8c601b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:51.603978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c72eecf-78dc-48f7-8048-98129a8c601b None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:51.610123 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 602/1219] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:10:51 2022] GET /identity/v3/role_assignments?user.id=fdacdcb2451742938164fdd44cdbaebe&scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb&role.id=4680aa64da80470b87075090fca2354b => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:10:53.664968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb34a4-e1c5-44c6-81cb-9edb98344a85 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:53.665504 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb34a4-e1c5-44c6-81cb-9edb98344a85 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:53.665616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb34a4-e1c5-44c6-81cb-9edb98344a85 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:53.668835 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 618/1220] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:10:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:10:53.676731 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b81928-03ce-4657-af3b-88e567f5d1f9 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:53.677252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b81928-03ce-4657-af3b-88e567f5d1f9 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:53.677367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b81928-03ce-4657-af3b-88e567f5d1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:53.696423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-33b81928-03ce-4657-af3b-88e567f5d1f9 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:10:53.697726 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-33b81928-03ce-4657-af3b-88e567f5d1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:10:53.769354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 603/1221] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:10:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:10:54.066554 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb34a4-e1c5-44c6-81cb-9edb98344a85 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:54.181436 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb34a4-e1c5-44c6-81cb-9edb98344a85 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:54.217761 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6116c6-d901-4558-9bee-64657b0d901a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:54.218390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6116c6-d901-4558-9bee-64657b0d901a None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:54.218665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6116c6-d901-4558-9bee-64657b0d901a None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:54.218928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6116c6-d901-4558-9bee-64657b0d901a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:54.229606 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6116c6-d901-4558-9bee-64657b0d901a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:10:54.234922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-ae6116c6-d901-4558-9bee-64657b0d901a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:10:54.234922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:10:54.234922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:10:54.234922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:10:54.234922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:10:54.234922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:10:54.234922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:10:54.234922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:10:54.234922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:10:54.234922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:10:54.234922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:10:54.234922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:10:54.234922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:10:54.234922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:10:54.234922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:10:54.234922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:10:54.236518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:10:54.236518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:10:54.236518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:10:54.236518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:10:54.236518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:10:54.236518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:10:54.236518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:10:54.236518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:10:54.236518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:10:54.236518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:10:54.236518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:10:54.236518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:10:54.236518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:10:54.236518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:10:54.236518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:10:54.236518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:10:54.237590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:10:54.237590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:10:54.237590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:10:54.237590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:10:54.237590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:10:54.237590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:10:54.237590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:10:54.237590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:10:54.237590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:10:54.237590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:10:54.237590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:10:54.237590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:10:54.239265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 619/1222] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 31 00:10:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:10:54.242342 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b81928-03ce-4657-af3b-88e567f5d1f9 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:54.361904 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b81928-03ce-4657-af3b-88e567f5d1f9 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:54.389288 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:10:54.403197 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf3d48-a6d3-4d3d-9db8-b5b3247774a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:54.403813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf3d48-a6d3-4d3d-9db8-b5b3247774a4 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:54.404092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf3d48-a6d3-4d3d-9db8-b5b3247774a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:54.404365 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf3d48-a6d3-4d3d-9db8-b5b3247774a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:54.405059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf3d48-a6d3-4d3d-9db8-b5b3247774a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:10:54.405367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf3d48-a6d3-4d3d-9db8-b5b3247774a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:54.406740 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf3d48-a6d3-4d3d-9db8-b5b3247774a4 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:54.416481 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 604/1223] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 31 00:10:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:10:54.422537 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6116c6-d901-4558-9bee-64657b0d901a None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:54.472764 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:10:54.540283 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6116c6-d901-4558-9bee-64657b0d901a None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:54.577943 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb3faf9-f8ce-4fce-a784-09b829553b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:54.578578 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3faf9-f8ce-4fce-a784-09b829553b3e None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:54.578859 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3faf9-f8ce-4fce-a784-09b829553b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:54.579166 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3faf9-f8ce-4fce-a784-09b829553b3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:54.590002 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb3faf9-f8ce-4fce-a784-09b829553b3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:54.591483 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb3faf9-f8ce-4fce-a784-09b829553b3e None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:54.612035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-8eb3faf9-f8ce-4fce-a784-09b829553b3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:10:54.612035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:10:54.612035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:10:54.612035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:10:54.612035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:10:54.612035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:10:54.612035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:10:54.612035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:10:54.612035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:10:54.612035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:10:54.612035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:10:54.612035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:10:54.612035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:10:54.612035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:10:54.612035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:10:54.612035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:10:54.613833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:10:54.613833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:10:54.613833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:10:54.613833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:10:54.613833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:10:54.613833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:10:54.613833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:10:54.613833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:10:54.613833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:10:54.613833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:10:54.613833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:10:54.613833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:10:54.613833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:10:54.613833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:10:54.613833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:10:54.613833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:10:54.613833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:10:54.614942 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:10:54.614942 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:10:54.614942 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:10:54.614942 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 620/1224] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:10:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:10:54.619611 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf3d48-a6d3-4d3d-9db8-b5b3247774a4 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:54.726239 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf3d48-a6d3-4d3d-9db8-b5b3247774a4 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:54.770521 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36404290-41e3-4269-ad7a-e1eba4c844b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:54.771306 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36404290-41e3-4269-ad7a-e1eba4c844b4 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:54.771587 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36404290-41e3-4269-ad7a-e1eba4c844b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:54.771854 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36404290-41e3-4269-ad7a-e1eba4c844b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:54.772507 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36404290-41e3-4269-ad7a-e1eba4c844b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:10:54.772913 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36404290-41e3-4269-ad7a-e1eba4c844b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:54.774348 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36404290-41e3-4269-ad7a-e1eba4c844b4 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:54.792154 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 605/1225] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:10:54.797897 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb3faf9-f8ce-4fce-a784-09b829553b3e None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:54.897060 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:10:54.904626 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb3faf9-f8ce-4fce-a784-09b829553b3e None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:54.944859 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd8906-50a8-401e-8771-98d6c8bd732a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:54.945491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd8906-50a8-401e-8771-98d6c8bd732a None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:54.945772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd8906-50a8-401e-8771-98d6c8bd732a None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:54.946044 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd8906-50a8-401e-8771-98d6c8bd732a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:54.953431 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd8906-50a8-401e-8771-98d6c8bd732a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:54.954857 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd8906-50a8-401e-8771-98d6c8bd732a None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:54.966947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-e2cd8906-50a8-401e-8771-98d6c8bd732a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 00:10:54.966947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:10:54.966947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:10:54.966947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:10:54.966947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:10:54.966947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:10:54.966947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:10:54.966947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:10:54.966947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:10:54.966947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:10:54.966947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:10:54.966947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:10:54.966947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:10:54.966947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:10:54.966947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:10:54.966947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:10:54.968049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:10:54.968049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:10:54.968049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:10:54.968049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:10:54.968049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:10:54.968049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:10:54.968049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:10:54.968049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:10:54.968049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:10:54.968049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:10:54.968049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:10:54.968049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:10:54.968049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:10:54.968049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:10:54.968049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:10:54.968049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:10:54.968049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:10:54.968049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:10:54.969023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:10:54.969023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:10:54.969023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:10:54.969023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:10:54.969023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:10:54.969023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:10:54.969023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:10:54.969023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:10:54.969023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 00:10:54.969023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:10:54.969023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 621/1226] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:10:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:10:54.975171 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36404290-41e3-4269-ad7a-e1eba4c844b4 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:55.036044 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:10:55.091911 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36404290-41e3-4269-ad7a-e1eba4c844b4 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:55.131260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcb153-a56f-4045-a47c-3f4f832ca908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:55.131896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcb153-a56f-4045-a47c-3f4f832ca908 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:55.132162 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcb153-a56f-4045-a47c-3f4f832ca908 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:55.132455 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcb153-a56f-4045-a47c-3f4f832ca908 None admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:55.133185 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcb153-a56f-4045-a47c-3f4f832ca908 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:10:55.133493 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcb153-a56f-4045-a47c-3f4f832ca908 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:55.135211 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcb153-a56f-4045-a47c-3f4f832ca908 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:55.155968 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 606/1227] 10.222.0.67 () {56 vars in 1143 bytes} [Wed Aug 31 00:10:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:10:55.162667 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd8906-50a8-401e-8771-98d6c8bd732a None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:55.273047 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd8906-50a8-401e-8771-98d6c8bd732a None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:55.310728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bcbfc-49c4-48e4-86c4-e8fc1a847bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:55.311520 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bcbfc-49c4-48e4-86c4-e8fc1a847bbb None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:55.311845 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bcbfc-49c4-48e4-86c4-e8fc1a847bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:55.312133 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bcbfc-49c4-48e4-86c4-e8fc1a847bbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:55.324955 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531bcbfc-49c4-48e4-86c4-e8fc1a847bbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:55.326724 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531bcbfc-49c4-48e4-86c4-e8fc1a847bbb None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:55.344125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-531bcbfc-49c4-48e4-86c4-e8fc1a847bbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:10:55.344125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:10:55.344125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:10:55.344125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:10:55.344125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:10:55.344125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:10:55.344125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:10:55.344125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:10:55.344125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:10:55.344125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:10:55.344125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:10:55.344125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:10:55.344125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:10:55.344125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:10:55.344125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:10:55.344125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:10:55.345424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:10:55.345424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:10:55.345424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:10:55.345424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:10:55.345424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:10:55.345424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:10:55.345424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:10:55.345424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:10:55.345424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:10:55.345424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:10:55.345424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:10:55.345424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:10:55.345424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:10:55.345424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:10:55.345424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:10:55.345424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:10:55.345424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:10:55.347583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:10:55.347583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:10:55.347583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:10:55.347583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 622/1228] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:10:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:10:55.352556 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcb153-a56f-4045-a47c-3f4f832ca908 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:55.466198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcb153-a56f-4045-a47c-3f4f832ca908 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:55.468608 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:10:55.510810 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6752bfc-ad46-4659-8fbf-89ee799c32db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:55.511461 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6752bfc-ad46-4659-8fbf-89ee799c32db None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:55.511741 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6752bfc-ad46-4659-8fbf-89ee799c32db None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:55.512014 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6752bfc-ad46-4659-8fbf-89ee799c32db None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:55.512733 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6752bfc-ad46-4659-8fbf-89ee799c32db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:10:55.513116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6752bfc-ad46-4659-8fbf-89ee799c32db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:55.514686 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6752bfc-ad46-4659-8fbf-89ee799c32db None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:55.528084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 607/1229] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:10:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:10:55.533826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bcbfc-49c4-48e4-86c4-e8fc1a847bbb None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:55.545639 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:10:55.653738 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bcbfc-49c4-48e4-86c4-e8fc1a847bbb None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:55.692071 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ed03d-6dfe-47dc-86c7-b0bd8d45a24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:55.692714 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ed03d-6dfe-47dc-86c7-b0bd8d45a24f None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:55.692999 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ed03d-6dfe-47dc-86c7-b0bd8d45a24f None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:55.693279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ed03d-6dfe-47dc-86c7-b0bd8d45a24f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:55.700217 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962ed03d-6dfe-47dc-86c7-b0bd8d45a24f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:55.701660 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962ed03d-6dfe-47dc-86c7-b0bd8d45a24f None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:55.714083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-962ed03d-6dfe-47dc-86c7-b0bd8d45a24f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:10:55.714083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:10:55.714083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:10:55.714083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:10:55.714083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:10:55.714083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:10:55.714083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:10:55.714083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:10:55.714083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:10:55.714083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:10:55.714083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:10:55.714083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:10:55.714083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:10:55.714083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:10:55.714083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:10:55.716549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:10:55.716549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:10:55.716549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:10:55.716549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:10:55.716549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:10:55.716549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:10:55.716549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:10:55.716549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:10:55.716549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:10:55.716549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:10:55.716549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:10:55.716549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:10:55.716549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:10:55.716549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:10:55.716549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:10:55.716549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:10:55.716549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:10:55.717801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:10:55.717801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:10:55.717801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:10:55.717801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:10:55.717801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:10:55.717801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:10:55.717801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:10:55.717801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:10:55.717801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 623/1230] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:10:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:10:55.723717 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6752bfc-ad46-4659-8fbf-89ee799c32db None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:55.842242 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6752bfc-ad46-4659-8fbf-89ee799c32db None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:55.880848 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a8ec0-c9b5-470f-ae13-181a04958702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:55.881485 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a8ec0-c9b5-470f-ae13-181a04958702 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:55.881739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a8ec0-c9b5-470f-ae13-181a04958702 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:55.882008 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a8ec0-c9b5-470f-ae13-181a04958702 None admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:55.882698 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a8ec0-c9b5-470f-ae13-181a04958702 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:10:55.883015 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a8ec0-c9b5-470f-ae13-181a04958702 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:55.884728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a8ec0-c9b5-470f-ae13-181a04958702 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:55.897963 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 608/1231] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:10:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:10:55.904194 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ed03d-6dfe-47dc-86c7-b0bd8d45a24f None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:55.978532 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:10:56.023293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ed03d-6dfe-47dc-86c7-b0bd8d45a24f None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:56.060033 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99d6bd-fc42-4005-aa6e-a0c41b540891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:56.060688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99d6bd-fc42-4005-aa6e-a0c41b540891 None admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:56.060964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99d6bd-fc42-4005-aa6e-a0c41b540891 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:56.061237 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99d6bd-fc42-4005-aa6e-a0c41b540891 None admin] PATH_INFO: `/v3/projects/cd1e6daca4264a0cbc11a9160d1213eb/users/fdacdcb2451742938164fdd44cdbaebe/roles/4680aa64da80470b87075090fca2354b` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:56.076501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99d6bd-fc42-4005-aa6e-a0c41b540891 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd1e6daca4264a0cbc11a9160d1213eb, user_id=fdacdcb2451742938164fdd44cdbaebe, role_id=4680aa64da80470b87075090fca2354b)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:56.077947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99d6bd-fc42-4005-aa6e-a0c41b540891 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:56.091690 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 624/1232] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 31 00:10:55 2022] PUT /identity/v3/projects/cd1e6daca4264a0cbc11a9160d1213eb/users/fdacdcb2451742938164fdd44cdbaebe/roles/4680aa64da80470b87075090fca2354b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:10:58.219631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3ae2a-010e-4149-8055-ee2ac0efe220 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:58.220179 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3ae2a-010e-4149-8055-ee2ac0efe220 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:58.220316 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3ae2a-010e-4149-8055-ee2ac0efe220 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:58.222553 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 609/1233] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:10:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:10:58.229055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30033e91-7f0d-457c-aabe-b0bba3034a77 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:58.229528 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30033e91-7f0d-457c-aabe-b0bba3034a77 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:58.229619 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30033e91-7f0d-457c-aabe-b0bba3034a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:58.249289 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-30033e91-7f0d-457c-aabe-b0bba3034a77 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:10:58.250424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-30033e91-7f0d-457c-aabe-b0bba3034a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:10:58.322682 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 625/1234] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:10:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:10:58.598896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab3ae2a-010e-4149-8055-ee2ac0efe220 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:58.624894 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:10:58.710781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab3ae2a-010e-4149-8055-ee2ac0efe220 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:58.746443 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfcafbe-b95a-40f2-bef2-15fa33a66b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:58.747065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfcafbe-b95a-40f2-bef2-15fa33a66b7d None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:58.747360 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfcafbe-b95a-40f2-bef2-15fa33a66b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:58.747614 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfcafbe-b95a-40f2-bef2-15fa33a66b7d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:58.758136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfcafbe-b95a-40f2-bef2-15fa33a66b7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:10:58.763205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-9bfcafbe-b95a-40f2-bef2-15fa33a66b7d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:10:58.763205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:10:58.763205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:10:58.763205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:10:58.763205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:10:58.763205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:10:58.763205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:10:58.763205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:10:58.763205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:10:58.763205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:10:58.763205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:10:58.763205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:10:58.763205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:10:58.763205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:10:58.763205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:10:58.763205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:10:58.764273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:10:58.764273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:10:58.764273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:10:58.764273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:10:58.764273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:10:58.764273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:10:58.764273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:10:58.764273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:10:58.764273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:10:58.764273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:10:58.764273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:10:58.764273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:10:58.764273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:10:58.764273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:10:58.764273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:10:58.764273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:10:58.765312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:10:58.765312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:10:58.765312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:10:58.765312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:10:58.765312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:10:58.765312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:10:58.765312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:10:58.765312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:10:58.765312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:10:58.765312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:10:58.765312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 00:10:58.765312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:10:58.767196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 610/1235] 10.222.0.67 () {56 vars in 1102 bytes} [Wed Aug 31 00:10:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:10:58.770821 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30033e91-7f0d-457c-aabe-b0bba3034a77 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:58.819262 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:10:58.879245 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30033e91-7f0d-457c-aabe-b0bba3034a77 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:58.917148 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104ead5-7881-49ee-8be0-25b79e212faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:58.917799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104ead5-7881-49ee-8be0-25b79e212faa None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:58.918077 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104ead5-7881-49ee-8be0-25b79e212faa None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:58.918349 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104ead5-7881-49ee-8be0-25b79e212faa None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:58.919020 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1104ead5-7881-49ee-8be0-25b79e212faa None admin] RBAC: Adding query filter params (name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:10:58.919351 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1104ead5-7881-49ee-8be0-25b79e212faa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:58.920719 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1104ead5-7881-49ee-8be0-25b79e212faa None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:58.929892 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 626/1236] 10.222.0.67 () {56 vars in 1103 bytes} [Wed Aug 31 00:10:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:10:58.935779 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfcafbe-b95a-40f2-bef2-15fa33a66b7d None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:59.032604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:10:59.049098 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfcafbe-b95a-40f2-bef2-15fa33a66b7d None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:59.095999 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49be9c-1416-43b4-a34d-fa64e4b3e41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:59.096630 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49be9c-1416-43b4-a34d-fa64e4b3e41e None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:59.096972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49be9c-1416-43b4-a34d-fa64e4b3e41e None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:59.097240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49be9c-1416-43b4-a34d-fa64e4b3e41e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:59.108026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49be9c-1416-43b4-a34d-fa64e4b3e41e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:59.109491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49be9c-1416-43b4-a34d-fa64e4b3e41e None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:59.124712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-4a49be9c-1416-43b4-a34d-fa64e4b3e41e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:10:59.124712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:10:59.124712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:10:59.124712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:10:59.124712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:10:59.124712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:10:59.124712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:10:59.124712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:10:59.124712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:10:59.124712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:10:59.124712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:10:59.124712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:10:59.124712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:10:59.124712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:10:59.124712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:10:59.124712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:10:59.126063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:10:59.126063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:10:59.126063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:10:59.126063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:10:59.126063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:10:59.126063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:10:59.126063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:10:59.126063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:10:59.126063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:10:59.126063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:10:59.126063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:10:59.126063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:10:59.126063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:10:59.126063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:10:59.126063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:10:59.126063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:10:59.126063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:10:59.127099 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:10:59.127099 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:10:59.127099 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:10:59.127099 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 611/1237] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:10:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:10:59.132663 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104ead5-7881-49ee-8be0-25b79e212faa None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:59.255835 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104ead5-7881-49ee-8be0-25b79e212faa None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:59.273301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:10:59.305833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bdab72-14a1-476f-a6f8-b69481e78be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:59.306448 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bdab72-14a1-476f-a6f8-b69481e78be6 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:59.306733 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bdab72-14a1-476f-a6f8-b69481e78be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:59.307009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bdab72-14a1-476f-a6f8-b69481e78be6 None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:59.307686 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bdab72-14a1-476f-a6f8-b69481e78be6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:10:59.307994 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bdab72-14a1-476f-a6f8-b69481e78be6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:59.309401 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bdab72-14a1-476f-a6f8-b69481e78be6 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:59.328257 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 627/1238] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:10:59.329480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49be9c-1416-43b4-a34d-fa64e4b3e41e None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:59.448520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49be9c-1416-43b4-a34d-fa64e4b3e41e None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:59.489753 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437de676-1347-4b9c-9970-bb5c1640d618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:59.490502 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437de676-1347-4b9c-9970-bb5c1640d618 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:59.490755 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437de676-1347-4b9c-9970-bb5c1640d618 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:59.491083 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437de676-1347-4b9c-9970-bb5c1640d618 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:59.494030 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:10:59.500307 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437de676-1347-4b9c-9970-bb5c1640d618 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:59.502172 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437de676-1347-4b9c-9970-bb5c1640d618 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:59.514834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-437de676-1347-4b9c-9970-bb5c1640d618 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 00:10:59.514834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:10:59.514834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:10:59.514834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:10:59.514834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:10:59.514834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:10:59.514834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:10:59.514834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:10:59.514834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:10:59.514834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:10:59.514834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:10:59.514834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:10:59.514834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:10:59.514834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:10:59.514834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:10:59.514834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:10:59.516116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:10:59.516116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:10:59.516116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:10:59.516116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:10:59.516116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:10:59.516116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:10:59.516116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:10:59.516116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:10:59.516116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:10:59.516116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:10:59.516116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:10:59.516116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:10:59.516116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:10:59.516116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:10:59.516116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:10:59.516116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:10:59.516116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:10:59.516116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:10:59.517081 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:10:59.517081 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:10:59.517081 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:10:59.517081 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:10:59.517081 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:10:59.517081 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:10:59.517081 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:10:59.517081 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:10:59.517081 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 00:10:59.517081 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:10:59.517081 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 612/1239] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:10:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:10:59.523254 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bdab72-14a1-476f-a6f8-b69481e78be6 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:59.652338 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bdab72-14a1-476f-a6f8-b69481e78be6 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:59.693134 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76868f1-89f3-4c36-af3b-0081e4becde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:59.693823 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76868f1-89f3-4c36-af3b-0081e4becde9 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:59.694131 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76868f1-89f3-4c36-af3b-0081e4becde9 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:59.694491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76868f1-89f3-4c36-af3b-0081e4becde9 None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:59.695659 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76868f1-89f3-4c36-af3b-0081e4becde9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:10:59.695949 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76868f1-89f3-4c36-af3b-0081e4becde9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:59.697729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76868f1-89f3-4c36-af3b-0081e4becde9 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:10:59.720649 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 628/1240] 10.222.0.67 () {56 vars in 1143 bytes} [Wed Aug 31 00:10:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:10:59.728433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437de676-1347-4b9c-9970-bb5c1640d618 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:10:59.850766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437de676-1347-4b9c-9970-bb5c1640d618 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:10:59.887978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fbaf5-4239-4d1c-9e31-33a8d03784f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:10:59.888588 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fbaf5-4239-4d1c-9e31-33a8d03784f2 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:10:59.888867 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fbaf5-4239-4d1c-9e31-33a8d03784f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:10:59.889138 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fbaf5-4239-4d1c-9e31-33a8d03784f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:10:59.959438 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:10:59.967204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396fbaf5-4239-4d1c-9e31-33a8d03784f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:10:59.970090 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396fbaf5-4239-4d1c-9e31-33a8d03784f2 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:00.087925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 613/1241] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:10:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:11:00.089385 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76868f1-89f3-4c36-af3b-0081e4becde9 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:00.119586 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:00.205066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76868f1-89f3-4c36-af3b-0081e4becde9 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:00.244386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100fc60-cdee-4f57-8f0f-83100f3911d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:00.244992 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100fc60-cdee-4f57-8f0f-83100f3911d2 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:00.245269 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100fc60-cdee-4f57-8f0f-83100f3911d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:00.245535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100fc60-cdee-4f57-8f0f-83100f3911d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:00.256738 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100fc60-cdee-4f57-8f0f-83100f3911d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:00.258194 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100fc60-cdee-4f57-8f0f-83100f3911d2 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:00.274948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-b100fc60-cdee-4f57-8f0f-83100f3911d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:11:00.274948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:11:00.274948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:11:00.274948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:11:00.274948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:11:00.274948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:11:00.274948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:11:00.274948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:11:00.274948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:11:00.274948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:11:00.274948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:11:00.274948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:11:00.274948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:11:00.274948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:11:00.274948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:11:00.274948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:11:00.276087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:11:00.276087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:11:00.276087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:11:00.276087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:11:00.276087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:11:00.276087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:11:00.276087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:11:00.276087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:11:00.276087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:11:00.276087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:11:00.276087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:11:00.276087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:11:00.276087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:11:00.276087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:11:00.276087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:11:00.276087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:11:00.276087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:11:00.277575 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:11:00.277575 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:11:00.277575 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:11:00.277575 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 629/1242] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:11:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:11:00.281938 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fbaf5-4239-4d1c-9e31-33a8d03784f2 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:00.388923 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:00.395903 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fbaf5-4239-4d1c-9e31-33a8d03784f2 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:00.433797 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a60f8-5cfb-47df-aa09-0ae5e4272823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:00.434533 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a60f8-5cfb-47df-aa09-0ae5e4272823 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:00.434913 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a60f8-5cfb-47df-aa09-0ae5e4272823 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:00.435283 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a60f8-5cfb-47df-aa09-0ae5e4272823 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:00.436195 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43a60f8-5cfb-47df-aa09-0ae5e4272823 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:11:00.436661 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43a60f8-5cfb-47df-aa09-0ae5e4272823 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:00.438673 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43a60f8-5cfb-47df-aa09-0ae5e4272823 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:00.452262 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 614/1243] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:11:00.458180 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100fc60-cdee-4f57-8f0f-83100f3911d2 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:00.553286 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:00.580205 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100fc60-cdee-4f57-8f0f-83100f3911d2 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:00.618226 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034c715-c88c-4715-956d-c66925c90f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:00.618853 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034c715-c88c-4715-956d-c66925c90f39 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:00.619167 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034c715-c88c-4715-956d-c66925c90f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:00.619440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034c715-c88c-4715-956d-c66925c90f39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:00.626330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034c715-c88c-4715-956d-c66925c90f39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:00.627813 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034c715-c88c-4715-956d-c66925c90f39 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:00.640301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-9034c715-c88c-4715-956d-c66925c90f39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:11:00.640301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:11:00.640301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:11:00.640301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:11:00.640301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:11:00.640301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:11:00.640301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:11:00.640301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:11:00.640301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:11:00.640301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:11:00.640301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:11:00.640301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:11:00.640301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:11:00.640301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:11:00.640301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:11:00.641264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:11:00.641264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:11:00.641264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:11:00.641264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:11:00.641264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:11:00.641264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:11:00.641264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:11:00.641264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:11:00.641264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:11:00.641264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:11:00.641264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:11:00.641264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:11:00.641264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:11:00.641264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:11:00.641264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:11:00.641264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:11:00.641264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:11:00.642531 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:11:00.642531 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:11:00.642531 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:11:00.642531 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:11:00.642531 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:11:00.642531 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:11:00.642531 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:11:00.642531 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:11:00.642531 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 630/1244] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:11:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:11:00.647466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a60f8-5cfb-47df-aa09-0ae5e4272823 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:00.760320 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a60f8-5cfb-47df-aa09-0ae5e4272823 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:00.800783 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcadec-c02f-4bb6-a28c-efc961c7612d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:00.801438 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcadec-c02f-4bb6-a28c-efc961c7612d None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:00.801748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcadec-c02f-4bb6-a28c-efc961c7612d None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:00.801957 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcadec-c02f-4bb6-a28c-efc961c7612d None admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:00.802652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcadec-c02f-4bb6-a28c-efc961c7612d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:11:00.802957 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcadec-c02f-4bb6-a28c-efc961c7612d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:00.804490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcadec-c02f-4bb6-a28c-efc961c7612d None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:00.817538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 615/1245] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:11:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:11:00.824341 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034c715-c88c-4715-956d-c66925c90f39 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:00.933945 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034c715-c88c-4715-956d-c66925c90f39 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:00.973361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be7c4e-2d96-42de-80a8-b5d3ec459026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:00.974019 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be7c4e-2d96-42de-80a8-b5d3ec459026 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:00.974337 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be7c4e-2d96-42de-80a8-b5d3ec459026 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:00.974661 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be7c4e-2d96-42de-80a8-b5d3ec459026 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:00.975514 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be7c4e-2d96-42de-80a8-b5d3ec459026 None admin] RBAC: Adding query filter params (role.id=4680aa64da80470b87075090fca2354b, scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb, user.id=fdacdcb2451742938164fdd44cdbaebe) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:11:00.975821 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be7c4e-2d96-42de-80a8-b5d3ec459026 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:00.977484 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be7c4e-2d96-42de-80a8-b5d3ec459026 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:00.993053 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 631/1246] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:11:00 2022] GET /identity/v3/role_assignments?user.id=fdacdcb2451742938164fdd44cdbaebe&scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb&role.id=4680aa64da80470b87075090fca2354b => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:11:03.079800 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd5e06-eda7-4182-b652-be22427c22cc None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:03.080611 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd5e06-eda7-4182-b652-be22427c22cc None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:03.080611 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd5e06-eda7-4182-b652-be22427c22cc None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:03.082912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 616/1247] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:11:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:11:03.088757 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb724b0-bffb-4e6d-9a57-734aa88afa88 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:03.089257 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb724b0-bffb-4e6d-9a57-734aa88afa88 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:03.089419 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb724b0-bffb-4e6d-9a57-734aa88afa88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:03.098355 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:03.113503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-0cb724b0-bffb-4e6d-9a57-734aa88afa88 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:11:03.114989 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-0cb724b0-bffb-4e6d-9a57-734aa88afa88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:11:03.189297 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 632/1248] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:11:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 00:11:03.473680 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd5e06-eda7-4182-b652-be22427c22cc None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:03.494408 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:03.596484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd5e06-eda7-4182-b652-be22427c22cc None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:03.634222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3763056c-ea78-4fdb-a2b6-53d858949079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:03.635139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763056c-ea78-4fdb-a2b6-53d858949079 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:03.635418 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763056c-ea78-4fdb-a2b6-53d858949079 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:03.635687 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763056c-ea78-4fdb-a2b6-53d858949079 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:03.649427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3763056c-ea78-4fdb-a2b6-53d858949079 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:11:03.655224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-3763056c-ea78-4fdb-a2b6-53d858949079 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:11:03.655224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:11:03.655224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:11:03.655224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:11:03.655224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:11:03.655224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:11:03.655224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:11:03.655224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:11:03.655224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:11:03.655224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:11:03.655224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:11:03.655224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:11:03.655224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:11:03.655224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:11:03.655224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:11:03.655224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:11:03.656411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:11:03.656411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:11:03.656411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:11:03.656411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:11:03.656411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:11:03.656411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:11:03.656411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:11:03.656411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:11:03.656411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:11:03.656411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:11:03.656411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:11:03.656411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:11:03.656411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:11:03.656411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:11:03.656411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:11:03.656411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:11:03.657516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:11:03.657516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:11:03.657516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:11:03.657516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:11:03.657516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:11:03.657516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:11:03.657516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:11:03.657516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:11:03.657516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:11:03.657516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:11:03.657516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:11:03.657516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:11:03.662459 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 617/1249] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:11:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:11:03.663786 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb724b0-bffb-4e6d-9a57-734aa88afa88 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:03.796285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb724b0-bffb-4e6d-9a57-734aa88afa88 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:03.830952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:03.837553 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e96bb8-0551-4811-b92c-61795770df71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:03.838166 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e96bb8-0551-4811-b92c-61795770df71 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:03.838446 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e96bb8-0551-4811-b92c-61795770df71 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:03.838714 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e96bb8-0551-4811-b92c-61795770df71 None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:03.839403 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e96bb8-0551-4811-b92c-61795770df71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:11:03.839705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e96bb8-0551-4811-b92c-61795770df71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:03.841235 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e96bb8-0551-4811-b92c-61795770df71 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:03.855971 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 633/1250] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:11:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:11:03.857180 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3763056c-ea78-4fdb-a2b6-53d858949079 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:03.943316 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:03.969798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3763056c-ea78-4fdb-a2b6-53d858949079 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:04.014982 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a5c85-a385-40fd-8efb-c21f8e90667c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:04.015657 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a5c85-a385-40fd-8efb-c21f8e90667c None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:04.015939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a5c85-a385-40fd-8efb-c21f8e90667c None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:04.016217 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a5c85-a385-40fd-8efb-c21f8e90667c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:04.028050 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a5c85-a385-40fd-8efb-c21f8e90667c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:04.029575 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a5c85-a385-40fd-8efb-c21f8e90667c None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:04.046622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-125a5c85-a385-40fd-8efb-c21f8e90667c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:11:04.046622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:11:04.046622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:11:04.046622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:11:04.046622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:11:04.046622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:11:04.046622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:11:04.046622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:11:04.046622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:11:04.046622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:11:04.046622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:11:04.046622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:11:04.046622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:11:04.046622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:11:04.046622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:11:04.046622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:11:04.047880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:11:04.047880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:11:04.047880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:11:04.047880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:11:04.047880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:11:04.047880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:11:04.047880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:11:04.047880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:11:04.047880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:11:04.047880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:11:04.047880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:11:04.047880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:11:04.047880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:11:04.047880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:11:04.047880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:11:04.047880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:11:04.047880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:11:04.049017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:11:04.049017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:11:04.049017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:11:04.049017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 618/1251] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:11:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:11:04.056047 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e96bb8-0551-4811-b92c-61795770df71 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:04.184197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e96bb8-0551-4811-b92c-61795770df71 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:04.224385 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002230c4-8bcc-458c-af36-27eb87110fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:04.225023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002230c4-8bcc-458c-af36-27eb87110fcf None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:04.225380 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002230c4-8bcc-458c-af36-27eb87110fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:04.225577 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002230c4-8bcc-458c-af36-27eb87110fcf None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:04.226239 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002230c4-8bcc-458c-af36-27eb87110fcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:11:04.226549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002230c4-8bcc-458c-af36-27eb87110fcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:04.227954 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002230c4-8bcc-458c-af36-27eb87110fcf None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:04.241272 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 634/1252] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:11:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:11:04.246886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a5c85-a385-40fd-8efb-c21f8e90667c None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:04.359545 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a5c85-a385-40fd-8efb-c21f8e90667c None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:04.392681 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:04.404291 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67692f32-04ce-4adc-a2ca-e29ce761ae36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:04.405027 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67692f32-04ce-4adc-a2ca-e29ce761ae36 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:04.405335 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67692f32-04ce-4adc-a2ca-e29ce761ae36 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:04.405617 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67692f32-04ce-4adc-a2ca-e29ce761ae36 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:04.415765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67692f32-04ce-4adc-a2ca-e29ce761ae36 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:04.417205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67692f32-04ce-4adc-a2ca-e29ce761ae36 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:04.430896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-67692f32-04ce-4adc-a2ca-e29ce761ae36 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 00:11:04.430896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:11:04.430896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:11:04.430896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:11:04.430896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:11:04.430896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:11:04.430896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:11:04.430896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:11:04.430896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:11:04.430896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:11:04.430896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:11:04.430896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:11:04.430896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:11:04.430896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:11:04.430896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:11:04.430896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:11:04.436122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:11:04.436122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:11:04.436122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:11:04.436122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:11:04.436122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:11:04.436122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:11:04.436122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:11:04.436122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:11:04.436122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:11:04.436122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:11:04.436122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:11:04.436122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:11:04.436122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:11:04.436122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:11:04.436122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:11:04.436122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:11:04.436122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:11:04.436122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:11:04.437599 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:11:04.437599 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:11:04.437599 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:11:04.437599 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:11:04.437599 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:11:04.437599 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:11:04.437599 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:11:04.437599 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:11:04.437599 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 00:11:04.437599 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:11:04.437599 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 619/1253] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:11:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:11:04.438268 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002230c4-8bcc-458c-af36-27eb87110fcf None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:04.505012 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:04.557991 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002230c4-8bcc-458c-af36-27eb87110fcf None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:04.595452 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8c7a7-6cbd-48cf-9dd2-b8c1b7920400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:04.596071 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8c7a7-6cbd-48cf-9dd2-b8c1b7920400 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:04.596342 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8c7a7-6cbd-48cf-9dd2-b8c1b7920400 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:04.596605 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8c7a7-6cbd-48cf-9dd2-b8c1b7920400 None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:04.598136 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8c7a7-6cbd-48cf-9dd2-b8c1b7920400 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:11:04.598503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8c7a7-6cbd-48cf-9dd2-b8c1b7920400 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:04.600012 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8c7a7-6cbd-48cf-9dd2-b8c1b7920400 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:04.617711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 635/1254] 10.222.0.67 () {56 vars in 1143 bytes} [Wed Aug 31 00:11:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:11:04.623829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67692f32-04ce-4adc-a2ca-e29ce761ae36 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:04.740466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67692f32-04ce-4adc-a2ca-e29ce761ae36 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:04.778315 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d82681-c411-450d-a98a-7e0b43eec678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:04.778930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d82681-c411-450d-a98a-7e0b43eec678 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:04.779238 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d82681-c411-450d-a98a-7e0b43eec678 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:04.779508 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d82681-c411-450d-a98a-7e0b43eec678 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:04.842822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:04.857025 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d82681-c411-450d-a98a-7e0b43eec678 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:04.858479 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d82681-c411-450d-a98a-7e0b43eec678 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:04.968809 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 620/1255] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:11:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 00:11:04.975056 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8c7a7-6cbd-48cf-9dd2-b8c1b7920400 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:05.088160 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8c7a7-6cbd-48cf-9dd2-b8c1b7920400 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:05.103183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:05.127152 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f494e7-4568-4312-8af7-702ae4bc4efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:05.127809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f494e7-4568-4312-8af7-702ae4bc4efc None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:05.128092 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f494e7-4568-4312-8af7-702ae4bc4efc None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:05.128384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f494e7-4568-4312-8af7-702ae4bc4efc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:05.140882 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f494e7-4568-4312-8af7-702ae4bc4efc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:05.142418 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f494e7-4568-4312-8af7-702ae4bc4efc None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:05.159174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-98f494e7-4568-4312-8af7-702ae4bc4efc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:11:05.159174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:11:05.159174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:11:05.159174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:11:05.159174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:11:05.159174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:11:05.159174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:11:05.159174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:11:05.159174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:11:05.159174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:11:05.159174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:11:05.159174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:11:05.159174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:11:05.159174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:11:05.159174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:11:05.159174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:11:05.161378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:11:05.161378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:11:05.161378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:11:05.161378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:11:05.161378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:11:05.161378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:11:05.161378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:11:05.161378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:11:05.161378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:11:05.161378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:11:05.161378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:11:05.161378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:11:05.161378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:11:05.161378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:11:05.161378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:11:05.161378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:11:05.161378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:11:05.162358 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:11:05.162358 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:11:05.162358 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:11:05.162358 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 636/1256] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:11:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:11:05.167424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d82681-c411-450d-a98a-7e0b43eec678 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:05.264357 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:05.278194 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d82681-c411-450d-a98a-7e0b43eec678 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:05.314309 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de8afec-bca2-4428-a299-e741ead97030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:05.314953 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8afec-bca2-4428-a299-e741ead97030 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:05.315263 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8afec-bca2-4428-a299-e741ead97030 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:05.315540 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8afec-bca2-4428-a299-e741ead97030 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:05.317202 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de8afec-bca2-4428-a299-e741ead97030 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:11:05.317474 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de8afec-bca2-4428-a299-e741ead97030 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:05.318887 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de8afec-bca2-4428-a299-e741ead97030 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:05.334672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 621/1257] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:11:05.340644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f494e7-4568-4312-8af7-702ae4bc4efc None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:05.453029 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f494e7-4568-4312-8af7-702ae4bc4efc None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:05.490110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfde292-a808-4aac-ab20-d1a1fbc6f8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:05.490751 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfde292-a808-4aac-ab20-d1a1fbc6f8d7 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:05.491143 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfde292-a808-4aac-ab20-d1a1fbc6f8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:05.491422 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfde292-a808-4aac-ab20-d1a1fbc6f8d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:05.497977 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfde292-a808-4aac-ab20-d1a1fbc6f8d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:05.499514 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfde292-a808-4aac-ab20-d1a1fbc6f8d7 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:05.512435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-edfde292-a808-4aac-ab20-d1a1fbc6f8d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:11:05.512435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:11:05.512435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:11:05.512435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:11:05.512435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:11:05.512435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:11:05.512435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:11:05.512435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:11:05.512435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:11:05.512435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:11:05.512435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:11:05.512435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:11:05.512435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:11:05.512435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:11:05.512435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:11:05.513556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:11:05.513556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:11:05.513556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:11:05.513556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:11:05.513556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:11:05.513556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:11:05.513556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:11:05.513556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:11:05.513556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:11:05.513556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:11:05.513556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:11:05.513556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:11:05.513556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:11:05.513556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:11:05.513556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:11:05.513556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:11:05.513556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:11:05.514673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:11:05.514673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:11:05.514673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:11:05.514673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:11:05.514673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:11:05.514673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:11:05.514673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:11:05.514673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:11:05.514673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 637/1258] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:11:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:11:05.522903 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de8afec-bca2-4428-a299-e741ead97030 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:05.639618 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de8afec-bca2-4428-a299-e741ead97030 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:05.675640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e1446-513a-4ef3-91a4-7006582e2307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:05.676244 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e1446-513a-4ef3-91a4-7006582e2307 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:05.676520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e1446-513a-4ef3-91a4-7006582e2307 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:05.676788 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e1446-513a-4ef3-91a4-7006582e2307 None admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:05.677471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e1446-513a-4ef3-91a4-7006582e2307 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:11:05.677772 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e1446-513a-4ef3-91a4-7006582e2307 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:05.679212 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e1446-513a-4ef3-91a4-7006582e2307 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:05.690947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 622/1259] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:11:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:11:05.696978 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfde292-a808-4aac-ab20-d1a1fbc6f8d7 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:05.726736 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:05.807199 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfde292-a808-4aac-ab20-d1a1fbc6f8d7 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:05.842572 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3224a-38db-443e-a98d-43f7e570d945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:05.843309 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3224a-38db-443e-a98d-43f7e570d945 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:05.843854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3224a-38db-443e-a98d-43f7e570d945 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:05.843854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3224a-38db-443e-a98d-43f7e570d945 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:05.844524 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db3224a-38db-443e-a98d-43f7e570d945 None admin] RBAC: Adding query filter params (role.id=34df06ce84ad4cf8b3bd1a4490228017, scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb, user.id=fdacdcb2451742938164fdd44cdbaebe) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:11:05.844874 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db3224a-38db-443e-a98d-43f7e570d945 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:05.846385 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db3224a-38db-443e-a98d-43f7e570d945 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:05.852622 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 638/1260] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:11:05 2022] GET /identity/v3/role_assignments?user.id=fdacdcb2451742938164fdd44cdbaebe&scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb&role.id=34df06ce84ad4cf8b3bd1a4490228017 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:11:07.798630 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799aec4-6449-4ddd-b4fb-4bfaa1f5208d None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:07.799204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799aec4-6449-4ddd-b4fb-4bfaa1f5208d None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:07.799204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799aec4-6449-4ddd-b4fb-4bfaa1f5208d None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:07.800637 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 623/1261] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:11:07.805805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ca217-33bd-41b8-b03c-c986d511fc91 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:07.806223 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ca217-33bd-41b8-b03c-c986d511fc91 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:07.806365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ca217-33bd-41b8-b03c-c986d511fc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:07.824261 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-a97ca217-33bd-41b8-b03c-c986d511fc91 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:11:07.825448 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a97ca217-33bd-41b8-b03c-c986d511fc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:11:07.890400 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 639/1262] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:11:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:11:08.161261 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a799aec4-6449-4ddd-b4fb-4bfaa1f5208d None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:08.172001 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:08.263805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a799aec4-6449-4ddd-b4fb-4bfaa1f5208d None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:08.301945 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091f209-526d-4e7f-885d-11d73c616a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:08.302560 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091f209-526d-4e7f-885d-11d73c616a8a None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:08.302833 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091f209-526d-4e7f-885d-11d73c616a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:08.303124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091f209-526d-4e7f-885d-11d73c616a8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:08.312838 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091f209-526d-4e7f-885d-11d73c616a8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:11:08.318073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-c091f209-526d-4e7f-885d-11d73c616a8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:11:08.318073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:11:08.318073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:11:08.318073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:11:08.318073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:11:08.318073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:11:08.318073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:11:08.318073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:11:08.318073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:11:08.318073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:11:08.318073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:11:08.318073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:11:08.318073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:11:08.318073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:11:08.318073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:11:08.318073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:11:08.319220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:11:08.319220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:11:08.319220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:11:08.319220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:11:08.319220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:11:08.319220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:11:08.319220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:11:08.319220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:11:08.319220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:11:08.319220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:11:08.319220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:11:08.319220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:11:08.319220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:11:08.319220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:11:08.319220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:11:08.319220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:11:08.320475 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:11:08.320475 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:11:08.320475 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:11:08.320475 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:11:08.320475 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:11:08.320475 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:11:08.320475 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:11:08.320475 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:11:08.320475 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:11:08.320475 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:11:08.320475 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:11:08.320475 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:11:08.320475 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 624/1263] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:11:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:11:08.325615 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ca217-33bd-41b8-b03c-c986d511fc91 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:08.347972 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:08.431317 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ca217-33bd-41b8-b03c-c986d511fc91 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:08.463928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24da59a-3877-4cf2-aa99-df78d7a647cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:08.464527 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24da59a-3877-4cf2-aa99-df78d7a647cb None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:08.464802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24da59a-3877-4cf2-aa99-df78d7a647cb None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:08.465069 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24da59a-3877-4cf2-aa99-df78d7a647cb None admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:08.465719 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24da59a-3877-4cf2-aa99-df78d7a647cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:11:08.466016 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24da59a-3877-4cf2-aa99-df78d7a647cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:08.467392 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24da59a-3877-4cf2-aa99-df78d7a647cb None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:08.475099 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 640/1264] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:11:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:11:08.480909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091f209-526d-4e7f-885d-11d73c616a8a None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:08.545181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:08.578528 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091f209-526d-4e7f-885d-11d73c616a8a None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:08.611504 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f0c0f0-8ba7-4a2b-80ff-43e232e82c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:08.612115 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0c0f0-8ba7-4a2b-80ff-43e232e82c0a None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:08.612389 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0c0f0-8ba7-4a2b-80ff-43e232e82c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:08.612659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0c0f0-8ba7-4a2b-80ff-43e232e82c0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:08.622752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f0c0f0-8ba7-4a2b-80ff-43e232e82c0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:08.624212 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f0c0f0-8ba7-4a2b-80ff-43e232e82c0a None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:08.637990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-b0f0c0f0-8ba7-4a2b-80ff-43e232e82c0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:11:08.637990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:11:08.637990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:11:08.637990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:11:08.637990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:11:08.637990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:11:08.637990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:11:08.637990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:11:08.637990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:11:08.637990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:11:08.637990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:11:08.637990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:11:08.637990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:11:08.637990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:11:08.637990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:11:08.637990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:11:08.639105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:11:08.639105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:11:08.639105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:11:08.639105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:11:08.639105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:11:08.639105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:11:08.639105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:11:08.639105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:11:08.639105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:11:08.639105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:11:08.639105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:11:08.639105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:11:08.639105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:11:08.639105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:11:08.639105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:11:08.639105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:11:08.639105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:11:08.640187 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:11:08.640187 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:11:08.640187 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:11:08.640187 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 625/1265] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:11:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:11:08.645460 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24da59a-3877-4cf2-aa99-df78d7a647cb None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:08.726445 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:08.744875 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24da59a-3877-4cf2-aa99-df78d7a647cb None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:08.779275 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e89aff-f0fa-4e96-83f0-92a35f32a3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:08.779890 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e89aff-f0fa-4e96-83f0-92a35f32a3df None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:08.780166 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e89aff-f0fa-4e96-83f0-92a35f32a3df None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:08.780436 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e89aff-f0fa-4e96-83f0-92a35f32a3df None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:08.781208 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e89aff-f0fa-4e96-83f0-92a35f32a3df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:11:08.781383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e89aff-f0fa-4e96-83f0-92a35f32a3df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:08.782758 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e89aff-f0fa-4e96-83f0-92a35f32a3df None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:08.799965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 641/1266] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:11:08.801003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f0c0f0-8ba7-4a2b-80ff-43e232e82c0a None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:08.903933 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f0c0f0-8ba7-4a2b-80ff-43e232e82c0a None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:08.924625 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:08.939737 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe2b7e-d7cb-4f76-86fa-cf857afb72e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:08.940369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe2b7e-d7cb-4f76-86fa-cf857afb72e0 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:08.940647 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe2b7e-d7cb-4f76-86fa-cf857afb72e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:08.940919 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe2b7e-d7cb-4f76-86fa-cf857afb72e0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:08.948653 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe2b7e-d7cb-4f76-86fa-cf857afb72e0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:08.950096 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe2b7e-d7cb-4f76-86fa-cf857afb72e0 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:08.961272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-5ebe2b7e-d7cb-4f76-86fa-cf857afb72e0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 00:11:08.961272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:11:08.961272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:11:08.961272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:11:08.961272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:11:08.961272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:11:08.961272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:11:08.961272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:11:08.961272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:11:08.961272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:11:08.961272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:11:08.961272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:11:08.961272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:11:08.961272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:11:08.961272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:11:08.961272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:11:08.962277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:11:08.962277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:11:08.962277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:11:08.962277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:11:08.962277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:11:08.962277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:11:08.962277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:11:08.962277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:11:08.962277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:11:08.962277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:11:08.962277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:11:08.962277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:11:08.962277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:11:08.962277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:11:08.962277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:11:08.962277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:11:08.962277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:11:08.962277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:11:08.963311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:11:08.963311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:11:08.963311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:11:08.963311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:11:08.963311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:11:08.963311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:11:08.963311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:11:08.963311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:11:08.963311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 00:11:08.963311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:11:08.963311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 626/1267] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:11:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 00:11:08.968246 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e89aff-f0fa-4e96-83f0-92a35f32a3df None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:09.070412 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e89aff-f0fa-4e96-83f0-92a35f32a3df None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:09.104497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72454d3-c4f6-4fd3-bf5b-2c7532f0df93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:09.105101 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72454d3-c4f6-4fd3-bf5b-2c7532f0df93 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:09.105383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72454d3-c4f6-4fd3-bf5b-2c7532f0df93 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:09.105658 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72454d3-c4f6-4fd3-bf5b-2c7532f0df93 None admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:09.106383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72454d3-c4f6-4fd3-bf5b-2c7532f0df93 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:11:09.106691 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72454d3-c4f6-4fd3-bf5b-2c7532f0df93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:09.108150 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72454d3-c4f6-4fd3-bf5b-2c7532f0df93 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:09.116663 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:09.125323 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 642/1268] 10.222.0.67 () {56 vars in 1143 bytes} [Wed Aug 31 00:11:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:11:09.131023 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe2b7e-d7cb-4f76-86fa-cf857afb72e0 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:09.228976 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe2b7e-d7cb-4f76-86fa-cf857afb72e0 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:09.262410 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b41379-9c0b-41d8-a56b-5b39e91ebbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:09.262957 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b41379-9c0b-41d8-a56b-5b39e91ebbaf None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:09.263262 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b41379-9c0b-41d8-a56b-5b39e91ebbaf None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:09.263534 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b41379-9c0b-41d8-a56b-5b39e91ebbaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:09.273721 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b41379-9c0b-41d8-a56b-5b39e91ebbaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:09.275176 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b41379-9c0b-41d8-a56b-5b39e91ebbaf None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:09.288912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-f8b41379-9c0b-41d8-a56b-5b39e91ebbaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:11:09.288912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:11:09.288912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:11:09.288912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:11:09.288912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:11:09.288912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:11:09.288912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:11:09.288912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:11:09.288912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:11:09.288912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:11:09.288912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:11:09.288912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:11:09.288912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:11:09.288912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:11:09.288912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:11:09.288912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:11:09.290026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:11:09.290026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:11:09.290026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:11:09.290026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:11:09.290026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:11:09.290026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:11:09.290026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:11:09.290026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:11:09.290026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:11:09.290026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:11:09.290026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:11:09.290026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:11:09.290026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:11:09.290026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:11:09.290026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:11:09.290026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:11:09.290026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:11:09.291127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:11:09.291127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:11:09.291127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:11:09.291127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 627/1269] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:11:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:11:09.295789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72454d3-c4f6-4fd3-bf5b-2c7532f0df93 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:09.406118 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72454d3-c4f6-4fd3-bf5b-2c7532f0df93 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:09.439575 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd122aba-002f-44c0-9aa0-4be2773a87f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:09.440175 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd122aba-002f-44c0-9aa0-4be2773a87f3 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:09.440454 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd122aba-002f-44c0-9aa0-4be2773a87f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:09.440727 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd122aba-002f-44c0-9aa0-4be2773a87f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:09.442027 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd122aba-002f-44c0-9aa0-4be2773a87f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:11:09.442331 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd122aba-002f-44c0-9aa0-4be2773a87f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:09.443725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd122aba-002f-44c0-9aa0-4be2773a87f3 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:09.454754 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 643/1270] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:11:09.460592 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b41379-9c0b-41d8-a56b-5b39e91ebbaf None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:09.492307 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:09.562631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b41379-9c0b-41d8-a56b-5b39e91ebbaf None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:09.596582 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae073ba0-35dc-4850-8960-384719e5b59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:09.597209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae073ba0-35dc-4850-8960-384719e5b59d None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:09.597562 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae073ba0-35dc-4850-8960-384719e5b59d None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:09.597883 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae073ba0-35dc-4850-8960-384719e5b59d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:09.604328 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae073ba0-35dc-4850-8960-384719e5b59d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:09.605767 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae073ba0-35dc-4850-8960-384719e5b59d None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:09.617545 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-ae073ba0-35dc-4850-8960-384719e5b59d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:11:09.617545 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:11:09.617545 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:11:09.617545 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:11:09.617545 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:11:09.617545 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:11:09.617545 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:11:09.617545 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:11:09.617545 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:11:09.617545 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:11:09.617545 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:11:09.617545 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:11:09.617545 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:11:09.617545 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:11:09.617545 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:11:09.618567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:11:09.618567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:11:09.618567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:11:09.618567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:11:09.618567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:11:09.618567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:11:09.618567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:11:09.618567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:11:09.618567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:11:09.618567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:11:09.618567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:11:09.618567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:11:09.618567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:11:09.618567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:11:09.618567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:11:09.618567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:11:09.618567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:11:09.619851 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:11:09.619851 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:11:09.619851 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:11:09.619851 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:11:09.619851 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:11:09.619851 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:11:09.619851 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:11:09.619851 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:11:09.619851 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 628/1271] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:11:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:11:09.624518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd122aba-002f-44c0-9aa0-4be2773a87f3 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:09.681994 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:09.723542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd122aba-002f-44c0-9aa0-4be2773a87f3 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:09.755938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70f39b-e9c3-428a-b976-eb36f669f759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:09.756541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70f39b-e9c3-428a-b976-eb36f669f759 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:09.756822 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70f39b-e9c3-428a-b976-eb36f669f759 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:09.757096 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70f39b-e9c3-428a-b976-eb36f669f759 None admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:09.757782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70f39b-e9c3-428a-b976-eb36f669f759 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:11:09.758088 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70f39b-e9c3-428a-b976-eb36f669f759 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:09.759542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70f39b-e9c3-428a-b976-eb36f669f759 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:09.771769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 644/1272] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:11:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:11:09.777894 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae073ba0-35dc-4850-8960-384719e5b59d None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:09.869634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:09.878933 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae073ba0-35dc-4850-8960-384719e5b59d None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:09.914101 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df7dc4-90e5-486f-9cf0-4865ec517624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:09.914747 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df7dc4-90e5-486f-9cf0-4865ec517624 None admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:09.915097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df7dc4-90e5-486f-9cf0-4865ec517624 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:09.915344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df7dc4-90e5-486f-9cf0-4865ec517624 None admin] PATH_INFO: `/v3/projects/cd1e6daca4264a0cbc11a9160d1213eb/users/fdacdcb2451742938164fdd44cdbaebe/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:09.917228 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df7dc4-90e5-486f-9cf0-4865ec517624 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd1e6daca4264a0cbc11a9160d1213eb, user_id=fdacdcb2451742938164fdd44cdbaebe, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:09.918640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df7dc4-90e5-486f-9cf0-4865ec517624 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:09.931976 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 629/1273] 10.222.0.67 () {58 vars in 1439 bytes} [Wed Aug 31 00:11:09 2022] PUT /identity/v3/projects/cd1e6daca4264a0cbc11a9160d1213eb/users/fdacdcb2451742938164fdd44cdbaebe/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:11:11.811731 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa5eb3-a9d7-4ad4-9baf-aa5526d397ae None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:11.812292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa5eb3-a9d7-4ad4-9baf-aa5526d397ae None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:11.812292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa5eb3-a9d7-4ad4-9baf-aa5526d397ae None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:11.813716 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 645/1274] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:11:11.820831 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab46631-584b-475d-ac73-b48fb1a53679 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:11.821322 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab46631-584b-475d-ac73-b48fb1a53679 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:11.821594 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab46631-584b-475d-ac73-b48fb1a53679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:11.844157 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-bab46631-584b-475d-ac73-b48fb1a53679 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:11:11.845325 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-bab46631-584b-475d-ac73-b48fb1a53679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:11:11.912589 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 630/1275] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:11:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:11:12.183745 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa5eb3-a9d7-4ad4-9baf-aa5526d397ae None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:12.283466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa5eb3-a9d7-4ad4-9baf-aa5526d397ae None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:12.300527 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:12.318540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b67dc3-96be-483f-bb77-abaa74597e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:12.319193 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b67dc3-96be-483f-bb77-abaa74597e1a None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:12.319477 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b67dc3-96be-483f-bb77-abaa74597e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:12.319750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b67dc3-96be-483f-bb77-abaa74597e1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:12.331526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b67dc3-96be-483f-bb77-abaa74597e1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 00:11:12.336875 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-94b67dc3-96be-483f-bb77-abaa74597e1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:11:12.336875 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:11:12.336875 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:11:12.336875 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:11:12.336875 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:11:12.336875 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:11:12.336875 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:11:12.336875 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:11:12.336875 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:11:12.336875 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:11:12.336875 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:11:12.336875 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:11:12.336875 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 00:11:12.336875 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 00:11:12.336875 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 00:11:12.336875 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 00:11:12.338183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 00:11:12.338183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 00:11:12.338183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 00:11:12.338183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 00:11:12.338183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:11:12.338183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:11:12.338183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:11:12.338183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:11:12.338183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:11:12.338183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:11:12.338183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:11:12.338183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:11:12.338183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:11:12.338183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:11:12.338183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:11:12.338183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:11:12.339242 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:11:12.339242 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:11:12.339242 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:11:12.339242 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:11:12.339242 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 00:11:12.339242 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 00:11:12.339242 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 00:11:12.339242 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 00:11:12.339242 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 00:11:12.339242 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 00:11:12.339242 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 00:11:12.339242 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:11:12.339242 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 646/1276] 10.222.0.67 () {56 vars in 1096 bytes} [Wed Aug 31 00:11:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:11:12.344129 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab46631-584b-475d-ac73-b48fb1a53679 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:12.424701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:12.444224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab46631-584b-475d-ac73-b48fb1a53679 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:12.476954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986cf111-9a8c-4be5-ad05-897332d72389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:12.477549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986cf111-9a8c-4be5-ad05-897332d72389 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:12.477823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986cf111-9a8c-4be5-ad05-897332d72389 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:12.478090 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986cf111-9a8c-4be5-ad05-897332d72389 None admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:12.478736 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986cf111-9a8c-4be5-ad05-897332d72389 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:11:12.479050 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986cf111-9a8c-4be5-ad05-897332d72389 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:12.480407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986cf111-9a8c-4be5-ad05-897332d72389 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:12.488894 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 631/1277] 10.222.0.67 () {56 vars in 1099 bytes} [Wed Aug 31 00:11:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:11:12.494550 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b67dc3-96be-483f-bb77-abaa74597e1a None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:12.593873 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b67dc3-96be-483f-bb77-abaa74597e1a None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:12.626727 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faae3a70-2bb1-4664-9374-982c5ede0316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:12.627444 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae3a70-2bb1-4664-9374-982c5ede0316 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:12.627728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae3a70-2bb1-4664-9374-982c5ede0316 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:12.628002 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae3a70-2bb1-4664-9374-982c5ede0316 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:12.638819 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faae3a70-2bb1-4664-9374-982c5ede0316 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:12.640295 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faae3a70-2bb1-4664-9374-982c5ede0316 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:12.655285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-faae3a70-2bb1-4664-9374-982c5ede0316 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:11:12.655285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:11:12.655285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:11:12.655285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:11:12.655285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:11:12.655285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:11:12.655285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:11:12.655285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:11:12.655285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:11:12.655285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:11:12.655285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:11:12.655285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:11:12.655285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:11:12.655285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:11:12.655285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:11:12.655285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:11:12.656386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:11:12.656386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:11:12.656386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:11:12.656386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:11:12.656386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:11:12.656386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:11:12.656386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:11:12.656386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:11:12.656386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:11:12.656386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:11:12.656386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:11:12.656386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:11:12.656386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:11:12.656386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:11:12.656386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:11:12.656386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:11:12.656386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:11:12.657540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:11:12.657540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:11:12.657540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:11:12.657540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 647/1278] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:11:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:11:12.662798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986cf111-9a8c-4be5-ad05-897332d72389 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:12.759982 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986cf111-9a8c-4be5-ad05-897332d72389 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:12.792327 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ceac9-b690-4eb0-bf24-4a92914a6962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:12.792922 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ceac9-b690-4eb0-bf24-4a92914a6962 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:12.793198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ceac9-b690-4eb0-bf24-4a92914a6962 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:12.793463 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ceac9-b690-4eb0-bf24-4a92914a6962 None admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:12.794088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ceac9-b690-4eb0-bf24-4a92914a6962 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:11:12.794387 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ceac9-b690-4eb0-bf24-4a92914a6962 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:12.795753 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ceac9-b690-4eb0-bf24-4a92914a6962 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:12.807657 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 632/1279] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:11:12.813541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faae3a70-2bb1-4664-9374-982c5ede0316 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:12.844781 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:12.916881 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faae3a70-2bb1-4664-9374-982c5ede0316 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:12.950076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bcf2ce-ff06-4e58-8f7e-3193ca8c02d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:12.950698 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcf2ce-ff06-4e58-8f7e-3193ca8c02d5 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:12.950979 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcf2ce-ff06-4e58-8f7e-3193ca8c02d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:12.951283 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcf2ce-ff06-4e58-8f7e-3193ca8c02d5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:12.959252 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bcf2ce-ff06-4e58-8f7e-3193ca8c02d5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:12.960701 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bcf2ce-ff06-4e58-8f7e-3193ca8c02d5 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:12.971513 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-36bcf2ce-ff06-4e58-8f7e-3193ca8c02d5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 00:11:12.971513 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:11:12.971513 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:11:12.971513 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:11:12.971513 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:11:12.971513 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:11:12.971513 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:11:12.971513 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:11:12.971513 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:11:12.971513 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:11:12.971513 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:11:12.971513 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:11:12.971513 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 00:11:12.971513 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 00:11:12.971513 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 00:11:12.971513 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 00:11:12.973322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:11:12.973322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:11:12.973322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 00:11:12.973322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:11:12.973322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 00:11:12.973322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 00:11:12.973322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:11:12.973322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:11:12.973322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:11:12.973322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:11:12.973322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:11:12.973322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:11:12.973322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:11:12.973322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:11:12.973322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:11:12.973322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:11:12.973322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:11:12.973322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:11:12.974551 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:11:12.974551 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:11:12.974551 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 00:11:12.974551 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 00:11:12.974551 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 00:11:12.974551 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 00:11:12.974551 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 00:11:12.974551 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 00:11:12.974551 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 00:11:12.974551 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:11:12.974551 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 648/1280] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:11:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:11:12.979372 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ceac9-b690-4eb0-bf24-4a92914a6962 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:12.983229 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:13.075432 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ceac9-b690-4eb0-bf24-4a92914a6962 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:13.108411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b65f4-c277-418f-a944-d961e9208873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:13.109004 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b65f4-c277-418f-a944-d961e9208873 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:13.109277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b65f4-c277-418f-a944-d961e9208873 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:13.109545 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b65f4-c277-418f-a944-d961e9208873 None admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:13.110259 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b65f4-c277-418f-a944-d961e9208873 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:11:13.110560 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b65f4-c277-418f-a944-d961e9208873 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:13.112000 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b65f4-c277-418f-a944-d961e9208873 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:13.128972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 633/1281] 10.222.0.67 () {56 vars in 1143 bytes} [Wed Aug 31 00:11:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:11:13.134967 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bcf2ce-ff06-4e58-8f7e-3193ca8c02d5 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:13.228249 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:13.237985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bcf2ce-ff06-4e58-8f7e-3193ca8c02d5 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:13.272274 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320aa8de-d71a-46ac-bc6d-40cd020a262d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:13.272870 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320aa8de-d71a-46ac-bc6d-40cd020a262d None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:13.273156 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320aa8de-d71a-46ac-bc6d-40cd020a262d None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:13.273432 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320aa8de-d71a-46ac-bc6d-40cd020a262d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:13.340567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320aa8de-d71a-46ac-bc6d-40cd020a262d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:13.342055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320aa8de-d71a-46ac-bc6d-40cd020a262d None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:13.429370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:13.439506 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 649/1282] 10.222.0.67 () {58 vars in 1282 bytes} [Wed Aug 31 00:11:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 00:11:13.445440 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b65f4-c277-418f-a944-d961e9208873 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:13.508875 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:13.546914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b65f4-c277-418f-a944-d961e9208873 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:13.580928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63538db9-7c89-45cc-bacc-718fdd38ca29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:13.581552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63538db9-7c89-45cc-bacc-718fdd38ca29 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:13.581831 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63538db9-7c89-45cc-bacc-718fdd38ca29 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:13.582736 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63538db9-7c89-45cc-bacc-718fdd38ca29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:13.592677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63538db9-7c89-45cc-bacc-718fdd38ca29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:13.594058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63538db9-7c89-45cc-bacc-718fdd38ca29 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:13.608800 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-63538db9-7c89-45cc-bacc-718fdd38ca29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:11:13.608800 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:11:13.608800 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:11:13.608800 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:11:13.608800 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:11:13.608800 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:11:13.608800 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:11:13.608800 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:11:13.608800 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:11:13.608800 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:11:13.608800 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:11:13.608800 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:11:13.608800 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 00:11:13.608800 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 00:11:13.608800 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 00:11:13.608800 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 00:11:13.609934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:11:13.609934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:11:13.609934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:11:13.609934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:11:13.609934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:11:13.609934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:11:13.609934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:11:13.609934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:11:13.609934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:11:13.609934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:11:13.609934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:11:13.609934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:11:13.609934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:11:13.609934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:11:13.609934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:11:13.609934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:11:13.609934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 00:11:13.610878 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 00:11:13.610878 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 00:11:13.610878 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:11:13.610878 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 634/1283] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:11:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:11:13.616289 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320aa8de-d71a-46ac-bc6d-40cd020a262d None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:13.712072 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320aa8de-d71a-46ac-bc6d-40cd020a262d None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:13.744691 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42764098-4f5d-4da6-b21f-eebe676777c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:13.745291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42764098-4f5d-4da6-b21f-eebe676777c9 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:13.745571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42764098-4f5d-4da6-b21f-eebe676777c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:13.745844 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42764098-4f5d-4da6-b21f-eebe676777c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:13.746475 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42764098-4f5d-4da6-b21f-eebe676777c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:11:13.746775 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42764098-4f5d-4da6-b21f-eebe676777c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:13.748160 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42764098-4f5d-4da6-b21f-eebe676777c9 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:13.759184 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 650/1284] 10.222.0.67 () {56 vars in 1109 bytes} [Wed Aug 31 00:11:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:11:13.765198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63538db9-7c89-45cc-bacc-718fdd38ca29 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:13.863780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63538db9-7c89-45cc-bacc-718fdd38ca29 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:13.877211 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:13.898308 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01861058-50fd-4c03-b8ea-4158161b2ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:13.898930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01861058-50fd-4c03-b8ea-4158161b2ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:13.899243 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01861058-50fd-4c03-b8ea-4158161b2ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:13.899519 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01861058-50fd-4c03-b8ea-4158161b2ca0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:13.905950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01861058-50fd-4c03-b8ea-4158161b2ca0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:13.907418 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01861058-50fd-4c03-b8ea-4158161b2ca0 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:13.919054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application [None req-01861058-50fd-4c03-b8ea-4158161b2ca0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:11:13.919054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:11:13.919054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:11:13.919054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:11:13.919054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:11:13.919054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:11:13.919054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:11:13.919054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:11:13.919054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:11:13.919054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:11:13.919054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:11:13.919054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:11:13.919054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 00:11:13.919054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 00:11:13.919054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 00:11:13.920084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 00:11:13.920084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:11:13.920084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:11:13.920084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:11:13.920084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:11:13.920084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:11:13.920084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:11:13.920084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:11:13.920084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application with Lock( Aug 31 00:11:13.920084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:11:13.920084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:11:13.920084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:11:13.920084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:11:13.920084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:11:13.920084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:11:13.920084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:11:13.920084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:11:13.921197 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 00:11:13.921197 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 00:11:13.921197 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 00:11:13.921197 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 00:11:13.921197 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 00:11:13.921197 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 00:11:13.921197 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 00:11:13.921197 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: ERROR keystone.server.flask.application Aug 31 00:11:13.921197 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 635/1285] 10.222.0.67 () {56 vars in 1111 bytes} [Wed Aug 31 00:11:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:11:13.926296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42764098-4f5d-4da6-b21f-eebe676777c9 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:13.976920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:14.028272 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42764098-4f5d-4da6-b21f-eebe676777c9 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:14.061009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a55b56-cdaa-4225-bf30-5a89f708ac67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:14.061611 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a55b56-cdaa-4225-bf30-5a89f708ac67 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:14.061888 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a55b56-cdaa-4225-bf30-5a89f708ac67 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:14.062158 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a55b56-cdaa-4225-bf30-5a89f708ac67 None admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:14.062846 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a55b56-cdaa-4225-bf30-5a89f708ac67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:11:14.063180 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a55b56-cdaa-4225-bf30-5a89f708ac67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:14.064607 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a55b56-cdaa-4225-bf30-5a89f708ac67 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:14.076960 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 651/1286] 10.222.0.67 () {56 vars in 1148 bytes} [Wed Aug 31 00:11:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:11:14.084017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01861058-50fd-4c03-b8ea-4158161b2ca0 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:14.181962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01861058-50fd-4c03-b8ea-4158161b2ca0 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:14.216352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c44c74-25e2-4281-a386-d03ed5280a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:14.216954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c44c74-25e2-4281-a386-d03ed5280a78 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:14.217243 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c44c74-25e2-4281-a386-d03ed5280a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:14.217516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c44c74-25e2-4281-a386-d03ed5280a78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:14.218246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c44c74-25e2-4281-a386-d03ed5280a78 None admin] RBAC: Adding query filter params (role.id=34df06ce84ad4cf8b3bd1a4490228017, scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb, user.id=fdacdcb2451742938164fdd44cdbaebe) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:11:14.218555 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c44c74-25e2-4281-a386-d03ed5280a78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:14.220006 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c44c74-25e2-4281-a386-d03ed5280a78 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:14.228215 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 636/1287] 10.222.0.67 () {56 vars in 1374 bytes} [Wed Aug 31 00:11:14 2022] GET /identity/v3/role_assignments?user.id=fdacdcb2451742938164fdd44cdbaebe&scope.project.id=cd1e6daca4264a0cbc11a9160d1213eb&role.id=34df06ce84ad4cf8b3bd1a4490228017 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:11:16.173100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5450063-7af7-4f67-a49e-85f9a3b9942c None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:16.173644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5450063-7af7-4f67-a49e-85f9a3b9942c None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:16.173644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5450063-7af7-4f67-a49e-85f9a3b9942c None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:16.174970 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 652/1288] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:11:16.180762 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b7bc0-58bd-43bb-a6de-ace7113d7c71 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:16.181209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b7bc0-58bd-43bb-a6de-ace7113d7c71 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:16.181303 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b7bc0-58bd-43bb-a6de-ace7113d7c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:16.199677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-0d1b7bc0-58bd-43bb-a6de-ace7113d7c71 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:11:16.200834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-0d1b7bc0-58bd-43bb-a6de-ace7113d7c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:11:16.229526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:16.267218 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 637/1289] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:11:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:11:16.565145 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5450063-7af7-4f67-a49e-85f9a3b9942c None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:16.667801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5450063-7af7-4f67-a49e-85f9a3b9942c None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:16.678813 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:16.702645 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df85d7a-d48f-48e7-a14d-470e44adbd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:16.703304 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df85d7a-d48f-48e7-a14d-470e44adbd06 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:16.703584 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df85d7a-d48f-48e7-a14d-470e44adbd06 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:16.703856 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df85d7a-d48f-48e7-a14d-470e44adbd06 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:16.704437 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df85d7a-d48f-48e7-a14d-470e44adbd06 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:16.705809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df85d7a-d48f-48e7-a14d-470e44adbd06 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:16.714678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-5df85d7a-d48f-48e7-a14d-470e44adbd06 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 00:11:16.714678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:11:16.714678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:11:16.714678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:11:16.714678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:11:16.714678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:11:16.714678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:11:16.714678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:11:16.714678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 00:11:16.714678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:11:16.714678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:11:16.714678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:11:16.714678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:11:16.714678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:11:16.714678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:11:16.714678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:11:16.714678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:11:16.715931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:11:16.715931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:11:16.715931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:11:16.715931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:11:16.715931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:11:16.715931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:11:16.715931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:11:16.715931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:11:16.715931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:11:16.715931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:11:16.715931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:11:16.715931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:11:16.715931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:11:16.715931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:11:16.715931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:11:16.715931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:11:16.716805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:11:16.716805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:11:16.716805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:11:16.716805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:11:16.716805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:11:16.716805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:11:16.716805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:11:16.716805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:11:16.716805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:11:16.716805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:11:16.716805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:11:16.716805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:11:16.716805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 00:11:16.716805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:11:16.716805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 653/1290] 10.222.0.67 () {56 vars in 1117 bytes} [Wed Aug 31 00:11:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:11:16.721915 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b7bc0-58bd-43bb-a6de-ace7113d7c71 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:16.818182 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b7bc0-58bd-43bb-a6de-ace7113d7c71 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:16.852254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27585173-fa8a-495b-a3da-2caf9bf7cceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:16.852854 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27585173-fa8a-495b-a3da-2caf9bf7cceb None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:16.853141 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27585173-fa8a-495b-a3da-2caf9bf7cceb None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:16.853402 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27585173-fa8a-495b-a3da-2caf9bf7cceb None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:16.854024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27585173-fa8a-495b-a3da-2caf9bf7cceb None admin] RBAC: Adding query filter params (name=placement) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:11:16.854323 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27585173-fa8a-495b-a3da-2caf9bf7cceb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:16.855716 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27585173-fa8a-495b-a3da-2caf9bf7cceb None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:16.860471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 638/1291] 10.222.0.67 () {56 vars in 1116 bytes} [Wed Aug 31 00:11:16 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:11:16.866862 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df85d7a-d48f-48e7-a14d-470e44adbd06 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:16.968715 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df85d7a-d48f-48e7-a14d-470e44adbd06 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:17.002237 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a1943-dda2-41b2-8aab-5f5e7130f4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:17.002837 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a1943-dda2-41b2-8aab-5f5e7130f4d8 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:17.003141 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a1943-dda2-41b2-8aab-5f5e7130f4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:17.003412 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a1943-dda2-41b2-8aab-5f5e7130f4d8 None admin] PATH_INFO: `/v3/services` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:17.004030 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a1943-dda2-41b2-8aab-5f5e7130f4d8 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:11:17.004832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a1943-dda2-41b2-8aab-5f5e7130f4d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:17.006199 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a1943-dda2-41b2-8aab-5f5e7130f4d8 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:17.010667 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 654/1292] 10.222.0.67 () {56 vars in 1116 bytes} [Wed Aug 31 00:11:16 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:11:18.909914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dd7b3-ede6-4b1b-a433-4d9cf9cbd7b4 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:18.910515 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dd7b3-ede6-4b1b-a433-4d9cf9cbd7b4 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:18.910667 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dd7b3-ede6-4b1b-a433-4d9cf9cbd7b4 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:18.912623 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 639/1293] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:11:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:11:18.918026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3a105-350c-4d72-8a64-2c244dd6a2c8 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:18.918457 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3a105-350c-4d72-8a64-2c244dd6a2c8 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:18.918573 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3a105-350c-4d72-8a64-2c244dd6a2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:18.937234 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-1ab3a105-350c-4d72-8a64-2c244dd6a2c8 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:11:18.938424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-1ab3a105-350c-4d72-8a64-2c244dd6a2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:11:18.971440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:19.009959 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 655/1294] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:11:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:11:19.280296 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59dd7b3-ede6-4b1b-a433-4d9cf9cbd7b4 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:19.310604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:19.392061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59dd7b3-ede6-4b1b-a433-4d9cf9cbd7b4 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:19.430411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435bd6ab-0332-4fec-872b-e84687161066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:19.431006 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bd6ab-0332-4fec-872b-e84687161066 None admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:19.431314 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bd6ab-0332-4fec-872b-e84687161066 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:19.431590 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bd6ab-0332-4fec-872b-e84687161066 None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:19.432233 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435bd6ab-0332-4fec-872b-e84687161066 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:19.433589 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435bd6ab-0332-4fec-872b-e84687161066 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:19.448379 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 640/1295] 10.222.0.67 () {60 vars in 1141 bytes} [Wed Aug 31 00:11:19 2022] POST /identity/v3/services => generated 246 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:11:21.368818 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a593764-f111-4ea1-b2fe-9f32b0b20c5d None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:21.369334 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a593764-f111-4ea1-b2fe-9f32b0b20c5d None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:21.369334 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a593764-f111-4ea1-b2fe-9f32b0b20c5d None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:21.370685 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 656/1296] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:11:21.375722 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e34e1-bdb6-43fe-97ca-c90c2829303a None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:21.376125 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e34e1-bdb6-43fe-97ca-c90c2829303a None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:21.376269 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e34e1-bdb6-43fe-97ca-c90c2829303a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:21.394831 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-e28e34e1-bdb6-43fe-97ca-c90c2829303a None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:11:21.396064 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-e28e34e1-bdb6-43fe-97ca-c90c2829303a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:11:21.466443 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 641/1297] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:11:21 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:11:21.733877 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a593764-f111-4ea1-b2fe-9f32b0b20c5d None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:21.838062 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a593764-f111-4ea1-b2fe-9f32b0b20c5d None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:21.871763 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1953fec-4728-4cda-af2d-9ef14da89851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:21.872389 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1953fec-4728-4cda-af2d-9ef14da89851 None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:21.872670 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1953fec-4728-4cda-af2d-9ef14da89851 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:21.872942 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1953fec-4728-4cda-af2d-9ef14da89851 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:21.873523 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1953fec-4728-4cda-af2d-9ef14da89851 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:21.874888 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1953fec-4728-4cda-af2d-9ef14da89851 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:21.884561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-f1953fec-4728-4cda-af2d-9ef14da89851 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 00:11:21.884561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:11:21.884561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:11:21.884561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:11:21.884561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:11:21.884561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:11:21.884561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:11:21.884561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:11:21.884561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 00:11:21.884561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:11:21.884561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:11:21.884561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:11:21.884561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:11:21.884561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:11:21.884561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:11:21.884561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:11:21.884561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:11:21.885609 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:11:21.885609 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:11:21.885609 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:11:21.885609 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:11:21.885609 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:11:21.885609 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:11:21.885609 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:11:21.885609 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:11:21.885609 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:11:21.885609 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:11:21.885609 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:11:21.885609 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:11:21.885609 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:11:21.885609 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:11:21.885609 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:11:21.885609 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:11:21.886713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:11:21.886713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:11:21.886713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:11:21.886713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:11:21.886713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:11:21.886713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:11:21.886713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:11:21.886713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:11:21.886713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:11:21.886713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:11:21.886713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:11:21.886713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:11:21.886713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 00:11:21.886713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:11:21.886713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 657/1298] 10.222.0.67 () {56 vars in 1117 bytes} [Wed Aug 31 00:11:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:11:21.891912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e34e1-bdb6-43fe-97ca-c90c2829303a None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:21.910173 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:21.991518 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e34e1-bdb6-43fe-97ca-c90c2829303a None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:22.026711 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebb125-ec29-443c-b6e0-91e79feab8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:22.027355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebb125-ec29-443c-b6e0-91e79feab8c5 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:22.027644 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebb125-ec29-443c-b6e0-91e79feab8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:22.027917 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebb125-ec29-443c-b6e0-91e79feab8c5 None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:22.028547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ebb125-ec29-443c-b6e0-91e79feab8c5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:11:22.028853 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ebb125-ec29-443c-b6e0-91e79feab8c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:22.030215 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ebb125-ec29-443c-b6e0-91e79feab8c5 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:22.035246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 642/1299] 10.222.0.67 () {56 vars in 1116 bytes} [Wed Aug 31 00:11:21 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:11:22.041301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1953fec-4728-4cda-af2d-9ef14da89851 None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:22.074005 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:22.142276 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1953fec-4728-4cda-af2d-9ef14da89851 None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:22.174731 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0121fb-99ad-4b2f-9ad7-154813c6468c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:22.175367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0121fb-99ad-4b2f-9ad7-154813c6468c None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:22.175652 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0121fb-99ad-4b2f-9ad7-154813c6468c None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:22.175926 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0121fb-99ad-4b2f-9ad7-154813c6468c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:22.176619 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0121fb-99ad-4b2f-9ad7-154813c6468c None admin] RBAC: Adding query filter params (interface=public, service_id=2b21d673c1844ed083f77a25fe7750b9, region_id=RegionOne) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:11:22.176930 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0121fb-99ad-4b2f-9ad7-154813c6468c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:22.179199 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0121fb-99ad-4b2f-9ad7-154813c6468c None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:22.184097 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 658/1300] 10.222.0.67 () {56 vars in 1251 bytes} [Wed Aug 31 00:11:22 2022] GET /identity/v3/endpoints?service_id=2b21d673c1844ed083f77a25fe7750b9&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:11:22.189791 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebb125-ec29-443c-b6e0-91e79feab8c5 None admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:22.275073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:22.293889 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebb125-ec29-443c-b6e0-91e79feab8c5 None admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:22.329196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cb911-eeef-4854-835f-ee9b91863a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:22.329804 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cb911-eeef-4854-835f-ee9b91863a92 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:22.330083 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cb911-eeef-4854-835f-ee9b91863a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:22.330356 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cb911-eeef-4854-835f-ee9b91863a92 None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:22.330959 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0cb911-eeef-4854-835f-ee9b91863a92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:22.332352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0cb911-eeef-4854-835f-ee9b91863a92 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:22.339273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 643/1301] 10.222.0.67 () {56 vars in 1087 bytes} [Wed Aug 31 00:11:22 2022] GET /identity/v3/services => generated 2191 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:11:24.245048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e278f2-579a-45de-99b1-26e309040568 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:24.245586 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e278f2-579a-45de-99b1-26e309040568 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:24.245586 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e278f2-579a-45de-99b1-26e309040568 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:24.246874 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 659/1302] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:11:24.251950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2d275-ff6d-4da4-a5b9-034f2cf1c90f None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:24.252319 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2d275-ff6d-4da4-a5b9-034f2cf1c90f None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:24.252503 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2d275-ff6d-4da4-a5b9-034f2cf1c90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:24.271260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-9ec2d275-ff6d-4da4-a5b9-034f2cf1c90f None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:11:24.272628 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-9ec2d275-ff6d-4da4-a5b9-034f2cf1c90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:11:24.340146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 644/1303] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:11:24 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 00:11:24.609432 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e278f2-579a-45de-99b1-26e309040568 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:24.701123 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:24.707938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e278f2-579a-45de-99b1-26e309040568 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:24.741079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6155aa6-a78b-4220-a3e8-cb7e4e3fa66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:24.741683 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6155aa6-a78b-4220-a3e8-cb7e4e3fa66c None admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:24.741959 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6155aa6-a78b-4220-a3e8-cb7e4e3fa66c None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:24.742224 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6155aa6-a78b-4220-a3e8-cb7e4e3fa66c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:24.742791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6155aa6-a78b-4220-a3e8-cb7e4e3fa66c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:24.744171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6155aa6-a78b-4220-a3e8-cb7e4e3fa66c None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:24.752594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application [None req-d6155aa6-a78b-4220-a3e8-cb7e4e3fa66c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 00:11:24.752594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:11:24.752594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 00:11:24.752594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 00:11:24.752594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 00:11:24.752594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 00:11:24.752594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 00:11:24.752594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:11:24.752594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 00:11:24.752594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:11:24.752594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 00:11:24.752594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:11:24.752594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 00:11:24.752594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 00:11:24.752594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 00:11:24.752594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 00:11:24.752594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 00:11:24.753774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 00:11:24.753774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 00:11:24.753774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 00:11:24.753774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 00:11:24.753774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 00:11:24.753774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 00:11:24.753774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 00:11:24.753774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 00:11:24.753774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 00:11:24.753774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 00:11:24.753774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 00:11:24.753774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 00:11:24.753774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 00:11:24.753774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 00:11:24.753774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 00:11:24.753774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 00:11:24.754620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 00:11:24.754620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application with Lock( Aug 31 00:11:24.754620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 00:11:24.754620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self._enter() Aug 31 00:11:24.754620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 00:11:24.754620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 00:11:24.754620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 00:11:24.754620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application return self.creator() Aug 31 00:11:24.754620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 00:11:24.754620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application created_value = creator( Aug 31 00:11:24.754620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 00:11:24.754620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 00:11:24.754620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 00:11:24.754620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: ERROR keystone.server.flask.application Aug 31 00:11:24.754620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 660/1304] 10.222.0.67 () {56 vars in 1117 bytes} [Wed Aug 31 00:11:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 00:11:24.759915 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2d275-ff6d-4da4-a5b9-034f2cf1c90f None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:24.832941 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:24.856947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2d275-ff6d-4da4-a5b9-034f2cf1c90f None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:24.890017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37fc1be-955c-4916-b027-082a10bdcf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:24.890555 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37fc1be-955c-4916-b027-082a10bdcf71 None admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:24.891114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37fc1be-955c-4916-b027-082a10bdcf71 None admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:24.891114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37fc1be-955c-4916-b027-082a10bdcf71 None admin] PATH_INFO: `/v3/services` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:24.891753 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37fc1be-955c-4916-b027-082a10bdcf71 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:11:24.892055 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37fc1be-955c-4916-b027-082a10bdcf71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:24.893411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37fc1be-955c-4916-b027-082a10bdcf71 None admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:24.899022 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 645/1305] 10.222.0.67 () {56 vars in 1116 bytes} [Wed Aug 31 00:11:24 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:11:24.904836 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6155aa6-a78b-4220-a3e8-cb7e4e3fa66c None admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:11:25.005407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6155aa6-a78b-4220-a3e8-cb7e4e3fa66c None admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:11:25.039554 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263aa3ea-dfe4-4a6f-a6a9-68385de73754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:11:25.040169 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263aa3ea-dfe4-4a6f-a6a9-68385de73754 None admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:11:25.040451 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263aa3ea-dfe4-4a6f-a6a9-68385de73754 None admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:11:25.040726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263aa3ea-dfe4-4a6f-a6a9-68385de73754 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:11:25.041396 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263aa3ea-dfe4-4a6f-a6a9-68385de73754 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:11:25.042759 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263aa3ea-dfe4-4a6f-a6a9-68385de73754 None admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:11:25.061328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 661/1306] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:11:24 2022] POST /identity/v3/endpoints => generated 331 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:11:34.895746 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=30 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:11:35.057073 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:12:30.346861 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c250f-b214-4866-aa01-4c7f91561150 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:30.347468 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c250f-b214-4866-aa01-4c7f91561150 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:30.347468 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c250f-b214-4866-aa01-4c7f91561150 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:30.348907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 646/1307] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:12:30.354483 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a24424-1791-4d01-b2fb-453d4269c1ec None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:30.355128 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a24424-1791-4d01-b2fb-453d4269c1ec None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:30.355519 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a24424-1791-4d01-b2fb-453d4269c1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:30.481248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-d7a24424-1791-4d01-b2fb-453d4269c1ec None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:12:30.482451 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-d7a24424-1791-4d01-b2fb-453d4269c1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:30.646840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 662/1308] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:12:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:12:30.974439 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1ef36-8464-4cf8-a882-772b50cd2d0d None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:30.974780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1ef36-8464-4cf8-a882-772b50cd2d0d None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:30.975174 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1ef36-8464-4cf8-a882-772b50cd2d0d None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:30.977388 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 647/1309] 10.222.0.67 () {54 vars in 965 bytes} [Wed Aug 31 00:12:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:12:30.986426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782291ff-952b-4fec-a0fe-7f9bcd459da8 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:30.986726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782291ff-952b-4fec-a0fe-7f9bcd459da8 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:30.986977 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782291ff-952b-4fec-a0fe-7f9bcd459da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:31.156774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-782291ff-952b-4fec-a0fe-7f9bcd459da8 None None] MFA Rules not processed for user `05c0d9554cf94c41a2cd709cab378e53`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:12:31.160393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-782291ff-952b-4fec-a0fe-7f9bcd459da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:31.203963 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:12:31.278803 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 663/1310] 10.222.0.67 () {58 vars in 1062 bytes} [Wed Aug 31 00:12:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:12:31.287506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1ef36-8464-4cf8-a882-772b50cd2d0d None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:12:31.289245 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-2cc1ef36-8464-4cf8-a882-772b50cd2d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:31.416352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1ef36-8464-4cf8-a882-772b50cd2d0d None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:12:31.510754 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fffc32-c403-4a97-9d74-79a1322bb605 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:12:31.511847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fffc32-c403-4a97-9d74-79a1322bb605 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:31.512315 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fffc32-c403-4a97-9d74-79a1322bb605 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:31.512592 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fffc32-c403-4a97-9d74-79a1322bb605 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:31.514294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-09fffc32-c403-4a97-9d74-79a1322bb605 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:31.580072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:12:31.650608 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fffc32-c403-4a97-9d74-79a1322bb605 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:12:31.652745 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fffc32-c403-4a97-9d74-79a1322bb605 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:12:31.675698 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 648/1311] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:12:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:12:33.930108 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ebc69-42bb-44c6-9f54-db88575ccc51 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:33.930675 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ebc69-42bb-44c6-9f54-db88575ccc51 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:33.930675 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ebc69-42bb-44c6-9f54-db88575ccc51 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:33.932025 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 664/1312] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:12:33.937650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82917ddd-34d1-491f-bede-c2b486b30f1b None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:33.938090 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82917ddd-34d1-491f-bede-c2b486b30f1b None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:33.938188 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82917ddd-34d1-491f-bede-c2b486b30f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:34.033694 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-82917ddd-34d1-491f-bede-c2b486b30f1b None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:12:34.034925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-82917ddd-34d1-491f-bede-c2b486b30f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:34.150712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 649/1313] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:12:34.439562 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ebc69-42bb-44c6-9f54-db88575ccc51 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:12:34.441263 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-1d8ebc69-42bb-44c6-9f54-db88575ccc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:34.546108 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ebc69-42bb-44c6-9f54-db88575ccc51 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:12:34.616321 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:12:34.619019 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db4819-4174-4795-97ae-fb81fdbed3a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:12:34.619871 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db4819-4174-4795-97ae-fb81fdbed3a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:34.620346 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db4819-4174-4795-97ae-fb81fdbed3a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:34.620658 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db4819-4174-4795-97ae-fb81fdbed3a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:34.622468 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-38db4819-4174-4795-97ae-fb81fdbed3a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:34.729278 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db4819-4174-4795-97ae-fb81fdbed3a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:12:34.731322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db4819-4174-4795-97ae-fb81fdbed3a5 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:12:34.749094 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 665/1314] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:12:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:12:37.004137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba6d32-1a0e-4558-bc39-8f0fe79ad5ca None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:37.004549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba6d32-1a0e-4558-bc39-8f0fe79ad5ca None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:37.004699 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba6d32-1a0e-4558-bc39-8f0fe79ad5ca None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:37.006108 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 650/1315] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:12:37.011247 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d7426-7da4-4c5e-ba68-701e365025e5 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:37.012277 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d7426-7da4-4c5e-ba68-701e365025e5 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:37.012571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d7426-7da4-4c5e-ba68-701e365025e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:37.083438 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-ae8d7426-7da4-4c5e-ba68-701e365025e5 None None] MFA Rules not processed for user `8eb6244b69ca487281ed852a375261ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:12:37.084658 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-ae8d7426-7da4-4c5e-ba68-701e365025e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:37.139704 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:12:37.191992 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 666/1316] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:12:37 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:12:37.476942 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba6d32-1a0e-4558-bc39-8f0fe79ad5ca None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:12:37.478505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-3bba6d32-1a0e-4558-bc39-8f0fe79ad5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:37.488922 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:12:37.581157 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba6d32-1a0e-4558-bc39-8f0fe79ad5ca None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:12:37.652949 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496f541-5580-4063-b0db-685341511a8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:12:37.653553 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496f541-5580-4063-b0db-685341511a8e service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:37.653834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496f541-5580-4063-b0db-685341511a8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:37.654104 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496f541-5580-4063-b0db-685341511a8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:37.655755 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b496f541-5580-4063-b0db-685341511a8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:37.717612 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:12:37.765020 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b496f541-5580-4063-b0db-685341511a8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:12:37.766391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b496f541-5580-4063-b0db-685341511a8e service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:12:37.785185 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 651/1317] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:12:37 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:12:40.261246 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044192b-a8ff-4fdd-ac4b-4369a0f795ee None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:40.261824 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044192b-a8ff-4fdd-ac4b-4369a0f795ee None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:40.261824 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044192b-a8ff-4fdd-ac4b-4369a0f795ee None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:40.263149 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 667/1318] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:12:40.268635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b711c7-d999-4b0b-87dd-1da49f20ae14 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:40.269055 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b711c7-d999-4b0b-87dd-1da49f20ae14 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:40.269212 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b711c7-d999-4b0b-87dd-1da49f20ae14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:40.336361 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-12b711c7-d999-4b0b-87dd-1da49f20ae14 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:12:40.337552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-12b711c7-d999-4b0b-87dd-1da49f20ae14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:40.435671 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 652/1319] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:12:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:12:40.725184 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044192b-a8ff-4fdd-ac4b-4369a0f795ee None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:12:40.726745 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-3044192b-a8ff-4fdd-ac4b-4369a0f795ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:40.834539 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044192b-a8ff-4fdd-ac4b-4369a0f795ee None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:12:40.908824 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0752e4-5e59-4649-b251-4041913e8e08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:12:40.909434 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0752e4-5e59-4649-b251-4041913e8e08 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:40.909715 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0752e4-5e59-4649-b251-4041913e8e08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:40.909989 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0752e4-5e59-4649-b251-4041913e8e08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:40.911743 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-6c0752e4-5e59-4649-b251-4041913e8e08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:40.919789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:12:41.023491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0752e4-5e59-4649-b251-4041913e8e08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:12:41.024931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0752e4-5e59-4649-b251-4041913e8e08 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:12:41.042497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 668/1320] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:12:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:12:43.641427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4885a-294f-4ead-b9fa-55cad32bb65b None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:43.641966 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4885a-294f-4ead-b9fa-55cad32bb65b None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:43.641966 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4885a-294f-4ead-b9fa-55cad32bb65b None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:43.643639 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 653/1321] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:12:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:12:43.648853 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b0e0d-589b-4995-88f7-c86a1546b29d None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:43.649256 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b0e0d-589b-4995-88f7-c86a1546b29d None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:43.649413 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b0e0d-589b-4995-88f7-c86a1546b29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:43.716590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-221b0e0d-589b-4995-88f7-c86a1546b29d None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:12:43.717787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-221b0e0d-589b-4995-88f7-c86a1546b29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:43.777995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:12:43.819221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 669/1322] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:12:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:12:44.112979 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4885a-294f-4ead-b9fa-55cad32bb65b None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:12:44.114589 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-88c4885a-294f-4ead-b9fa-55cad32bb65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:44.134508 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:12:44.221990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4885a-294f-4ead-b9fa-55cad32bb65b None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:12:44.296651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abf500-bdd4-44a9-847e-e1513eb409c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:12:44.297175 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abf500-bdd4-44a9-847e-e1513eb409c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:44.297455 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abf500-bdd4-44a9-847e-e1513eb409c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:44.297725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abf500-bdd4-44a9-847e-e1513eb409c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:44.299341 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-13abf500-bdd4-44a9-847e-e1513eb409c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:44.377809 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:12:44.417681 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abf500-bdd4-44a9-847e-e1513eb409c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:12:44.419265 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abf500-bdd4-44a9-847e-e1513eb409c3 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:12:44.437890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 654/1323] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:12:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:12:46.452816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138cc65-cf14-429b-b214-04e64297f7c1 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:46.453389 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138cc65-cf14-429b-b214-04e64297f7c1 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:46.453389 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138cc65-cf14-429b-b214-04e64297f7c1 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:46.454716 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 670/1324] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:12:46.460045 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc796e75-9d96-4aac-aba5-3cdc958f0194 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:46.460437 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc796e75-9d96-4aac-aba5-3cdc958f0194 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:46.460717 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc796e75-9d96-4aac-aba5-3cdc958f0194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:46.527196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-cc796e75-9d96-4aac-aba5-3cdc958f0194 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:12:46.528988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-cc796e75-9d96-4aac-aba5-3cdc958f0194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:46.624223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 655/1325] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:12:46.908167 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8138cc65-cf14-429b-b214-04e64297f7c1 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:12:46.909713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-8138cc65-cf14-429b-b214-04e64297f7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:47.025842 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8138cc65-cf14-429b-b214-04e64297f7c1 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:12:47.098460 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f70e4-3870-4511-a596-2e116e62cf2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:12:47.099105 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f70e4-3870-4511-a596-2e116e62cf2f service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:47.099470 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f70e4-3870-4511-a596-2e116e62cf2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:47.099661 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f70e4-3870-4511-a596-2e116e62cf2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:47.101459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-942f70e4-3870-4511-a596-2e116e62cf2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:47.120135 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:12:47.217701 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942f70e4-3870-4511-a596-2e116e62cf2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:12:47.219086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942f70e4-3870-4511-a596-2e116e62cf2f service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:12:47.237299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 671/1326] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:12:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:12:49.243355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d1064-a998-4117-adb5-eac989a4738b None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:49.243904 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d1064-a998-4117-adb5-eac989a4738b None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:49.243904 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d1064-a998-4117-adb5-eac989a4738b None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:49.245270 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 656/1327] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:12:49.250727 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb6087-9a38-4907-86f7-86f9554528d9 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:49.251259 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb6087-9a38-4907-86f7-86f9554528d9 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:49.251541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb6087-9a38-4907-86f7-86f9554528d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:49.331880 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-50eb6087-9a38-4907-86f7-86f9554528d9 None None] MFA Rules not processed for user `8eb6244b69ca487281ed852a375261ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:12:49.333109 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-50eb6087-9a38-4907-86f7-86f9554528d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:49.410521 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:12:49.447144 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 672/1328] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:12:49 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:12:49.744085 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d1064-a998-4117-adb5-eac989a4738b None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:12:49.745625 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c67d1064-a998-4117-adb5-eac989a4738b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:49.789757 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:12:49.885862 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d1064-a998-4117-adb5-eac989a4738b None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:12:49.978227 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df76579-8a82-4071-b152-2743605b9913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:12:49.978765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df76579-8a82-4071-b152-2743605b9913 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:49.979074 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df76579-8a82-4071-b152-2743605b9913 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:49.979357 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df76579-8a82-4071-b152-2743605b9913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:49.981028 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-6df76579-8a82-4071-b152-2743605b9913 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:50.072190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:12:50.111261 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df76579-8a82-4071-b152-2743605b9913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:12:50.112983 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df76579-8a82-4071-b152-2743605b9913 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:12:50.133419 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 657/1329] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:12:49 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:12:52.332087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48383c3-5ed5-4879-b637-4e46cdc63fc3 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:52.333019 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48383c3-5ed5-4879-b637-4e46cdc63fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:52.333019 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48383c3-5ed5-4879-b637-4e46cdc63fc3 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:52.335868 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 673/1330] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:12:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:12:52.342269 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb39ace-ec93-46b5-9756-c87574e6d11c None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:52.342823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb39ace-ec93-46b5-9756-c87574e6d11c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:52.342823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb39ace-ec93-46b5-9756-c87574e6d11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:52.412833 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-8cb39ace-ec93-46b5-9756-c87574e6d11c None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:12:52.414150 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-8cb39ace-ec93-46b5-9756-c87574e6d11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:52.520362 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 658/1331] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:12:52.813065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48383c3-5ed5-4879-b637-4e46cdc63fc3 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:12:52.814618 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e48383c3-5ed5-4879-b637-4e46cdc63fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:52.916485 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48383c3-5ed5-4879-b637-4e46cdc63fc3 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:12:52.988286 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf3d72-68ca-402f-bf3b-5a8df659111f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:12:52.988877 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf3d72-68ca-402f-bf3b-5a8df659111f service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:52.989157 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf3d72-68ca-402f-bf3b-5a8df659111f service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:52.989424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf3d72-68ca-402f-bf3b-5a8df659111f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:52.991053 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-29cf3d72-68ca-402f-bf3b-5a8df659111f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:53.019600 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:12:53.102067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf3d72-68ca-402f-bf3b-5a8df659111f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:12:53.103548 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf3d72-68ca-402f-bf3b-5a8df659111f service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:12:53.121766 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 674/1332] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:12:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:12:55.119566 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ce1eb-26a9-454f-9233-bb187fd2f650 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:55.120088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ce1eb-26a9-454f-9233-bb187fd2f650 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:55.120184 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ce1eb-26a9-454f-9233-bb187fd2f650 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:55.121511 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 659/1333] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:12:55.127345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffee257-9452-4d4f-a3d8-40fe28a0706e None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:55.127780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffee257-9452-4d4f-a3d8-40fe28a0706e None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:55.127902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffee257-9452-4d4f-a3d8-40fe28a0706e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:55.194549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-8ffee257-9452-4d4f-a3d8-40fe28a0706e None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:12:55.195772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-8ffee257-9452-4d4f-a3d8-40fe28a0706e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:55.270695 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:12:55.292760 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 675/1334] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:12:55.583277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ce1eb-26a9-454f-9233-bb187fd2f650 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:12:55.584819 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-de9ce1eb-26a9-454f-9233-bb187fd2f650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:55.626091 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:12:55.690934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ce1eb-26a9-454f-9233-bb187fd2f650 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:12:55.764150 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300dcf8c-a67a-47a6-ab44-aa839482dc64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:12:55.764844 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dcf8c-a67a-47a6-ab44-aa839482dc64 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:55.765180 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dcf8c-a67a-47a6-ab44-aa839482dc64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:55.765514 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300dcf8c-a67a-47a6-ab44-aa839482dc64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:55.767188 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-300dcf8c-a67a-47a6-ab44-aa839482dc64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:55.850254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:12:55.876649 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300dcf8c-a67a-47a6-ab44-aa839482dc64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:12:55.878074 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300dcf8c-a67a-47a6-ab44-aa839482dc64 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:12:55.896555 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 660/1335] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:12:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:12:58.152499 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb5e77-8aaf-4c3f-8f50-9f0aa7bf8b7c None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:58.153057 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb5e77-8aaf-4c3f-8f50-9f0aa7bf8b7c None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:58.153057 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb5e77-8aaf-4c3f-8f50-9f0aa7bf8b7c None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:58.156878 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 676/1336] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:12:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:12:58.163681 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88eec3-f68d-4774-9464-34720d785103 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:58.164125 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88eec3-f68d-4774-9464-34720d785103 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:58.164245 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88eec3-f68d-4774-9464-34720d785103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:58.234333 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-4f88eec3-f68d-4774-9464-34720d785103 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:12:58.235602 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-4f88eec3-f68d-4774-9464-34720d785103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:58.333201 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 661/1337] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:12:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:12:58.613192 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb5e77-8aaf-4c3f-8f50-9f0aa7bf8b7c None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:12:58.614746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-8afb5e77-8aaf-4c3f-8f50-9f0aa7bf8b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:58.719656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb5e77-8aaf-4c3f-8f50-9f0aa7bf8b7c None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:12:58.798839 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14892c4d-d429-4d42-8ec0-64f4a91731fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:12:58.799471 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14892c4d-d429-4d42-8ec0-64f4a91731fa service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:12:58.799755 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14892c4d-d429-4d42-8ec0-64f4a91731fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:12:58.800025 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14892c4d-d429-4d42-8ec0-64f4a91731fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:12:58.801620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-14892c4d-d429-4d42-8ec0-64f4a91731fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:12:58.837207 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:12:58.909478 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14892c4d-d429-4d42-8ec0-64f4a91731fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:12:58.910830 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14892c4d-d429-4d42-8ec0-64f4a91731fa service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:12:58.932866 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 677/1338] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:12:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:13:02.821620 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665f452-d33e-4c12-9eaf-752857088fca None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:02.822161 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665f452-d33e-4c12-9eaf-752857088fca None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:02.822331 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665f452-d33e-4c12-9eaf-752857088fca None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:02.823805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 662/1339] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:13:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:13:02.829332 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b12b6e-7cd4-4d6e-99ce-2ea8539e075b None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:02.829768 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b12b6e-7cd4-4d6e-99ce-2ea8539e075b None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:02.829893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b12b6e-7cd4-4d6e-99ce-2ea8539e075b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:02.897357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-d5b12b6e-7cd4-4d6e-99ce-2ea8539e075b None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:13:02.898565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-d5b12b6e-7cd4-4d6e-99ce-2ea8539e075b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:02.988464 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:03.003176 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 678/1340] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:13:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:13:03.301478 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716d0b1-6a9a-45a9-abd1-470258a12a98 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:03.301763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716d0b1-6a9a-45a9-abd1-470258a12a98 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:03.302037 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716d0b1-6a9a-45a9-abd1-470258a12a98 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:03.303423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 663/1341] 10.222.0.67 () {54 vars in 965 bytes} [Wed Aug 31 00:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:13:03.309430 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452b36f-b993-478c-8252-e5d2723e3a85 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:03.309720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452b36f-b993-478c-8252-e5d2723e3a85 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:03.310552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452b36f-b993-478c-8252-e5d2723e3a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:03.390601 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-9452b36f-b993-478c-8252-e5d2723e3a85 None None] MFA Rules not processed for user `05c0d9554cf94c41a2cd709cab378e53`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:13:03.391832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-9452b36f-b993-478c-8252-e5d2723e3a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:03.469368 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 679/1342] 10.222.0.67 () {58 vars in 1062 bytes} [Wed Aug 31 00:13:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:13:03.478353 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716d0b1-6a9a-45a9-abd1-470258a12a98 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:13:03.480005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-8716d0b1-6a9a-45a9-abd1-470258a12a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:03.529246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:03.587215 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716d0b1-6a9a-45a9-abd1-470258a12a98 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:13:03.663230 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15554e-e50b-4c1b-adac-d99f7643b513 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:13:03.663839 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15554e-e50b-4c1b-adac-d99f7643b513 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:03.664115 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15554e-e50b-4c1b-adac-d99f7643b513 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:03.664448 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15554e-e50b-4c1b-adac-d99f7643b513 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:03.666205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-1d15554e-e50b-4c1b-adac-d99f7643b513 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:03.760137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:03.778003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15554e-e50b-4c1b-adac-d99f7643b513 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:13:03.780054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15554e-e50b-4c1b-adac-d99f7643b513 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:13:03.798574 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 664/1343] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:13:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:13:06.952932 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d5664-132e-4503-8060-38c8f7bbd0fc None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:06.953488 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d5664-132e-4503-8060-38c8f7bbd0fc None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:06.953488 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d5664-132e-4503-8060-38c8f7bbd0fc None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:06.954817 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 680/1344] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:13:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:13:06.960046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f45d40-18d8-48ee-ad1a-d117420dbb01 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:06.960467 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f45d40-18d8-48ee-ad1a-d117420dbb01 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:06.960583 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f45d40-18d8-48ee-ad1a-d117420dbb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:07.029300 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-13f45d40-18d8-48ee-ad1a-d117420dbb01 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:13:07.030463 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-13f45d40-18d8-48ee-ad1a-d117420dbb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:07.135138 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 665/1345] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:13:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:13:07.433850 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d5664-132e-4503-8060-38c8f7bbd0fc None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:13:07.436097 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-675d5664-132e-4503-8060-38c8f7bbd0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:07.532517 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:07.551021 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d5664-132e-4503-8060-38c8f7bbd0fc None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:13:07.624673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e430bb-25a5-4d67-aeb8-2e4672abb493 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:13:07.625296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e430bb-25a5-4d67-aeb8-2e4672abb493 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:07.625658 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e430bb-25a5-4d67-aeb8-2e4672abb493 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:07.625838 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e430bb-25a5-4d67-aeb8-2e4672abb493 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:07.627555 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-42e430bb-25a5-4d67-aeb8-2e4672abb493 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:07.761792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e430bb-25a5-4d67-aeb8-2e4672abb493 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:13:07.763553 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e430bb-25a5-4d67-aeb8-2e4672abb493 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:13:07.787487 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 681/1346] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:13:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:13:12.012082 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4617def-5a27-4dba-96a9-b8ff82ddb457 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:12.012645 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4617def-5a27-4dba-96a9-b8ff82ddb457 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:12.012728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4617def-5a27-4dba-96a9-b8ff82ddb457 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:12.014180 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 666/1347] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:13:12.020122 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03fd1d-6609-4f47-8c38-9f8082c87b82 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:12.020649 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03fd1d-6609-4f47-8c38-9f8082c87b82 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:12.020649 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03fd1d-6609-4f47-8c38-9f8082c87b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:12.037139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:12.093073 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-3c03fd1d-6609-4f47-8c38-9f8082c87b82 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:13:12.094435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-3c03fd1d-6609-4f47-8c38-9f8082c87b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:12.199892 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 682/1348] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:13:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:13:12.485597 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4617def-5a27-4dba-96a9-b8ff82ddb457 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:13:12.487187 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b4617def-5a27-4dba-96a9-b8ff82ddb457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:12.549502 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:12.597664 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4617def-5a27-4dba-96a9-b8ff82ddb457 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:13:12.677358 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7f085-02c6-4601-8612-66d261a899b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:13:12.678019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7f085-02c6-4601-8612-66d261a899b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:12.678297 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7f085-02c6-4601-8612-66d261a899b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:12.678571 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7f085-02c6-4601-8612-66d261a899b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:12.680287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-0da7f085-02c6-4601-8612-66d261a899b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:12.789443 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:12.793732 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7f085-02c6-4601-8612-66d261a899b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:13:12.795986 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7f085-02c6-4601-8612-66d261a899b3 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:13:12.814901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 667/1349] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:13:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:13:17.801511 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1a101-ff32-4513-aa00-3bb5396c20ee None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:17.802020 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1a101-ff32-4513-aa00-3bb5396c20ee None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:17.802100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1a101-ff32-4513-aa00-3bb5396c20ee None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:17.803776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 683/1350] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 31 00:13:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:13:17.809861 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ef032-ca2c-4283-86b9-4f9cd41508ce None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:17.810300 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ef032-ca2c-4283-86b9-4f9cd41508ce None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:17.810449 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ef032-ca2c-4283-86b9-4f9cd41508ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:17.906955 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-a94ef032-ca2c-4283-86b9-4f9cd41508ce None None] MFA Rules not processed for user `fdacdcb2451742938164fdd44cdbaebe`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:13:17.908553 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-a94ef032-ca2c-4283-86b9-4f9cd41508ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:18.066080 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 668/1351] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 31 00:13:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:13:22.198044 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=59 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:22.311221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5ed20-dd8a-4dea-94cb-85aa2c04400b None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:22.312491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5ed20-dd8a-4dea-94cb-85aa2c04400b None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:22.313140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5ed20-dd8a-4dea-94cb-85aa2c04400b None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:22.314685 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 684/1352] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 31 00:13:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:13:22.319970 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77b809-7240-467f-9ea5-c4f86f506434 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:22.320455 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77b809-7240-467f-9ea5-c4f86f506434 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:22.320675 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77b809-7240-467f-9ea5-c4f86f506434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:22.383477 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:22.408499 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-cb77b809-7240-467f-9ea5-c4f86f506434 None None] MFA Rules not processed for user `fdacdcb2451742938164fdd44cdbaebe`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:13:22.409819 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-cb77b809-7240-467f-9ea5-c4f86f506434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:22.532415 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 669/1353] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 31 00:13:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:13:24.088824 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56024c06-1b57-4991-af8c-95ffcfd66e3f None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:24.089700 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56024c06-1b57-4991-af8c-95ffcfd66e3f None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:24.089982 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56024c06-1b57-4991-af8c-95ffcfd66e3f None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:24.091396 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 685/1354] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 31 00:13:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:13:24.098019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352edd9-af0e-42ff-9d61-8990702d26b8 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:24.098502 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352edd9-af0e-42ff-9d61-8990702d26b8 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:24.098601 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352edd9-af0e-42ff-9d61-8990702d26b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:24.189109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-0352edd9-af0e-42ff-9d61-8990702d26b8 None None] MFA Rules not processed for user `fdacdcb2451742938164fdd44cdbaebe`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:13:24.190321 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-0352edd9-af0e-42ff-9d61-8990702d26b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:24.236450 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:24.305076 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 670/1355] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 31 00:13:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:13:27.662020 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f6d4f-b2b5-49ba-ad21-83adac018623 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:27.662630 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f6d4f-b2b5-49ba-ad21-83adac018623 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:27.662630 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f6d4f-b2b5-49ba-ad21-83adac018623 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:27.664139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 686/1356] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:13:27.669125 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50240baa-b443-4437-9fa0-6c100dbed93d None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:27.669849 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50240baa-b443-4437-9fa0-6c100dbed93d None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:27.670396 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50240baa-b443-4437-9fa0-6c100dbed93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:27.749791 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-50240baa-b443-4437-9fa0-6c100dbed93d None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:13:27.753748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-50240baa-b443-4437-9fa0-6c100dbed93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:27.890747 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 671/1357] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:13:28.221901 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9651b14-1a30-4a1a-8d5f-32579bfde323 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:28.222915 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9651b14-1a30-4a1a-8d5f-32579bfde323 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:28.223876 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9651b14-1a30-4a1a-8d5f-32579bfde323 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:28.227834 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 687/1358] 10.222.0.67 () {54 vars in 962 bytes} [Wed Aug 31 00:13:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:13:28.235955 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4d76d-8306-41be-8de3-69151faf9dd4 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:28.236871 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4d76d-8306-41be-8de3-69151faf9dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:28.237445 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4d76d-8306-41be-8de3-69151faf9dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:28.257504 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:28.336462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-2ed4d76d-8306-41be-8de3-69151faf9dd4 None None] MFA Rules not processed for user `3aa53ae5a30146f69d7d6eefe4290dba`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:13:28.337658 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-2ed4d76d-8306-41be-8de3-69151faf9dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:28.474225 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 672/1359] 10.222.0.67 () {58 vars in 1059 bytes} [Wed Aug 31 00:13:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:13:28.482038 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9651b14-1a30-4a1a-8d5f-32579bfde323 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:13:28.483575 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e9651b14-1a30-4a1a-8d5f-32579bfde323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:28.653898 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9651b14-1a30-4a1a-8d5f-32579bfde323 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:13:28.731107 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:28.773525 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e7349-7eb6-4c3a-8cd2-728b0e4b5c24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:13:28.774141 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e7349-7eb6-4c3a-8cd2-728b0e4b5c24 service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:28.774440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e7349-7eb6-4c3a-8cd2-728b0e4b5c24 service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:28.774704 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e7349-7eb6-4c3a-8cd2-728b0e4b5c24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:28.776349 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-bf4e7349-7eb6-4c3a-8cd2-728b0e4b5c24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:28.872283 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e7349-7eb6-4c3a-8cd2-728b0e4b5c24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:13:28.873924 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e7349-7eb6-4c3a-8cd2-728b0e4b5c24 service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:13:28.893063 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 688/1360] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:13:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:13:31.060816 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9d897-1e71-40b6-bca9-7ef37954f8c9 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:31.061312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9d897-1e71-40b6-bca9-7ef37954f8c9 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:31.061312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9d897-1e71-40b6-bca9-7ef37954f8c9 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:31.062851 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 673/1361] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:13:31.067964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c35cf-d239-456f-94cd-051c6d77fe41 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:31.068388 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c35cf-d239-456f-94cd-051c6d77fe41 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:31.068511 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c35cf-d239-456f-94cd-051c6d77fe41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:31.142495 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-f31c35cf-d239-456f-94cd-051c6d77fe41 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:13:31.144056 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f31c35cf-d239-456f-94cd-051c6d77fe41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:31.192778 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:31.267365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 689/1362] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:13:31.561528 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9d897-1e71-40b6-bca9-7ef37954f8c9 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:13:31.562792 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-37d9d897-1e71-40b6-bca9-7ef37954f8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:31.637424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:31.729377 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9d897-1e71-40b6-bca9-7ef37954f8c9 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:13:31.853598 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79996eb-1e6d-4296-9f31-6a6d2b7f27de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:13:31.854590 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79996eb-1e6d-4296-9f31-6a6d2b7f27de service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:31.855088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79996eb-1e6d-4296-9f31-6a6d2b7f27de service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:31.855542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79996eb-1e6d-4296-9f31-6a6d2b7f27de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:31.857578 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f79996eb-1e6d-4296-9f31-6a6d2b7f27de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:31.883367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:31.965925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79996eb-1e6d-4296-9f31-6a6d2b7f27de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:13:31.967789 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79996eb-1e6d-4296-9f31-6a6d2b7f27de service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:13:31.990345 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 674/1363] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:13:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:13:34.267500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def147f7-164e-41c7-ad0c-940ccc420e23 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:34.268016 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def147f7-164e-41c7-ad0c-940ccc420e23 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:34.268016 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def147f7-164e-41c7-ad0c-940ccc420e23 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:34.269635 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 690/1364] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:13:34.275806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b29c5-593a-4371-8b7e-d16a4810d203 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:34.276268 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b29c5-593a-4371-8b7e-d16a4810d203 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:34.276541 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b29c5-593a-4371-8b7e-d16a4810d203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:34.360729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-b93b29c5-593a-4371-8b7e-d16a4810d203 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:13:34.362077 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b93b29c5-593a-4371-8b7e-d16a4810d203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:34.475744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:34.483234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 675/1365] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:13:34.777626 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ad1be-b865-45bc-8b00-62c9ce3b504f None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:34.777905 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ad1be-b865-45bc-8b00-62c9ce3b504f None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:34.778357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ad1be-b865-45bc-8b00-62c9ce3b504f None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:34.779740 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 691/1366] 10.222.0.67 () {54 vars in 962 bytes} [Wed Aug 31 00:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:13:34.786225 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b95d4-8778-47e1-806f-eebc895bb5ea None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:34.786792 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b95d4-8778-47e1-806f-eebc895bb5ea None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:34.787205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b95d4-8778-47e1-806f-eebc895bb5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:34.869341 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-487b95d4-8778-47e1-806f-eebc895bb5ea None None] MFA Rules not processed for user `3aa53ae5a30146f69d7d6eefe4290dba`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:13:34.870887 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-487b95d4-8778-47e1-806f-eebc895bb5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:35.005187 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 676/1367] 10.222.0.67 () {58 vars in 1059 bytes} [Wed Aug 31 00:13:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:13:35.012947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ad1be-b865-45bc-8b00-62c9ce3b504f None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:13:35.014976 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-836ad1be-b865-45bc-8b00-62c9ce3b504f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:35.180220 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ad1be-b865-45bc-8b00-62c9ce3b504f None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:13:35.203907 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:35.310021 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75b4e4-3539-48c2-abdd-be62e9fabb24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:13:35.310820 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75b4e4-3539-48c2-abdd-be62e9fabb24 service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:35.311068 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75b4e4-3539-48c2-abdd-be62e9fabb24 service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:35.311378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75b4e4-3539-48c2-abdd-be62e9fabb24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:35.314966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-3d75b4e4-3539-48c2-abdd-be62e9fabb24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:35.416529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d75b4e4-3539-48c2-abdd-be62e9fabb24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:13:35.418399 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d75b4e4-3539-48c2-abdd-be62e9fabb24 service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:13:35.438869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 692/1368] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:13:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:13:36.061700 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ff498-cac6-4c2d-9c79-f40c0334c557 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:36.062733 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ff498-cac6-4c2d-9c79-f40c0334c557 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:36.062834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ff498-cac6-4c2d-9c79-f40c0334c557 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:36.064950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 677/1369] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:13:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:13:36.070177 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420473b-576f-4e86-b0d0-0591d9946d56 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:36.070466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420473b-576f-4e86-b0d0-0591d9946d56 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:36.070738 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420473b-576f-4e86-b0d0-0591d9946d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:36.104926 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:36.144276 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-1420473b-576f-4e86-b0d0-0591d9946d56 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:13:36.145502 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-1420473b-576f-4e86-b0d0-0591d9946d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:36.255164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 693/1370] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:13:36.846872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0635df-306c-46c1-80b6-3b5fc520dde5 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:36.846872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0635df-306c-46c1-80b6-3b5fc520dde5 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:36.846872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0635df-306c-46c1-80b6-3b5fc520dde5 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:36.848515 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 678/1371] 10.222.0.67 () {54 vars in 964 bytes} [Wed Aug 31 00:13:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:13:36.853296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbefa67-be86-4e2f-81fb-10d322456fea None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:36.853801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbefa67-be86-4e2f-81fb-10d322456fea None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:36.853987 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbefa67-be86-4e2f-81fb-10d322456fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:36.931567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-1cbefa67-be86-4e2f-81fb-10d322456fea None None] MFA Rules not processed for user `7e42dfb7d22e4435889cb91327db5477`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:13:36.932849 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-1cbefa67-be86-4e2f-81fb-10d322456fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:36.979765 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:37.013111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 694/1372] 10.222.0.67 () {58 vars in 1061 bytes} [Wed Aug 31 00:13:36 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:13:37.021864 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0635df-306c-46c1-80b6-3b5fc520dde5 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:13:37.023539 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-cc0635df-306c-46c1-80b6-3b5fc520dde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:37.076169 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:37.135414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0635df-306c-46c1-80b6-3b5fc520dde5 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:13:37.217641 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b9a30-5dc5-480d-9957-3067d608052b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:13:37.218369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b9a30-5dc5-480d-9957-3067d608052b service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:37.219560 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b9a30-5dc5-480d-9957-3067d608052b service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:37.220278 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b9a30-5dc5-480d-9957-3067d608052b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:37.222038 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-909b9a30-5dc5-480d-9957-3067d608052b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:37.341769 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:37.354287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b9a30-5dc5-480d-9957-3067d608052b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:13:37.355847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b9a30-5dc5-480d-9957-3067d608052b service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:13:37.377705 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 679/1373] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:13:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:13:37.486298 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff1788-575e-4ae2-ab3b-2e4a3c88aba3 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:37.486589 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff1788-575e-4ae2-ab3b-2e4a3c88aba3 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:37.486869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff1788-575e-4ae2-ab3b-2e4a3c88aba3 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:37.488587 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 695/1374] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:13:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:13:37.493482 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe421e-aac1-4697-9675-27b6c6a68560 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:37.493759 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe421e-aac1-4697-9675-27b6c6a68560 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:37.494033 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe421e-aac1-4697-9675-27b6c6a68560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:37.587089 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-57fe421e-aac1-4697-9675-27b6c6a68560 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:13:37.588524 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-57fe421e-aac1-4697-9675-27b6c6a68560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:37.712043 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 680/1375] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:13:38.007782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff1788-575e-4ae2-ab3b-2e4a3c88aba3 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:13:38.009436 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-19ff1788-575e-4ae2-ab3b-2e4a3c88aba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:38.165592 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff1788-575e-4ae2-ab3b-2e4a3c88aba3 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:13:38.217429 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:38.292493 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222d5f1-cd96-40d9-8e55-9e8017a2d722 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:13:38.293010 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222d5f1-cd96-40d9-8e55-9e8017a2d722 service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:38.293292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222d5f1-cd96-40d9-8e55-9e8017a2d722 service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:38.293561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222d5f1-cd96-40d9-8e55-9e8017a2d722 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:38.295240 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-6222d5f1-cd96-40d9-8e55-9e8017a2d722 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:38.386278 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6222d5f1-cd96-40d9-8e55-9e8017a2d722 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:13:38.387942 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6222d5f1-cd96-40d9-8e55-9e8017a2d722 service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:13:38.406438 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 696/1376] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:13:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:13:40.397703 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05eefea-36a6-4967-a6db-bfb387e1e57d None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:40.398252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05eefea-36a6-4967-a6db-bfb387e1e57d None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:40.398252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05eefea-36a6-4967-a6db-bfb387e1e57d None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:40.399675 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 681/1377] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:13:40.405111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa319c-0053-4a13-9254-ef420d6f1b57 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:40.405533 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa319c-0053-4a13-9254-ef420d6f1b57 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:40.405646 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa319c-0053-4a13-9254-ef420d6f1b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:40.473717 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-f2aa319c-0053-4a13-9254-ef420d6f1b57 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:13:40.474949 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f2aa319c-0053-4a13-9254-ef420d6f1b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:40.490567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:40.612543 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 697/1378] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:13:40.897799 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05eefea-36a6-4967-a6db-bfb387e1e57d None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:13:40.899381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c05eefea-36a6-4967-a6db-bfb387e1e57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:40.976275 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:41.091561 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05eefea-36a6-4967-a6db-bfb387e1e57d None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:13:41.226016 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce718e60-666e-4010-9d66-b6e261ce4b64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:13:41.226892 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce718e60-666e-4010-9d66-b6e261ce4b64 service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:41.227370 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce718e60-666e-4010-9d66-b6e261ce4b64 service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:41.227810 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce718e60-666e-4010-9d66-b6e261ce4b64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:41.229750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-ce718e60-666e-4010-9d66-b6e261ce4b64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:41.250749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:41.338025 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce718e60-666e-4010-9d66-b6e261ce4b64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:13:41.339747 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce718e60-666e-4010-9d66-b6e261ce4b64 service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:13:41.361266 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 682/1379] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:13:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:13:43.348247 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28fe7c-9012-478c-b77f-f577fcd06120 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:43.348801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28fe7c-9012-478c-b77f-f577fcd06120 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:43.348801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28fe7c-9012-478c-b77f-f577fcd06120 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:43.350271 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 698/1380] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:13:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:13:43.355243 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03362125-0636-48fd-a77d-efee499fe976 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:43.355636 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03362125-0636-48fd-a77d-efee499fe976 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:43.355798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03362125-0636-48fd-a77d-efee499fe976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:43.424648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-03362125-0636-48fd-a77d-efee499fe976 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:13:43.425835 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-03362125-0636-48fd-a77d-efee499fe976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:43.527133 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:43.535179 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 683/1381] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:13:43.825856 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28fe7c-9012-478c-b77f-f577fcd06120 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:13:43.827563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4c28fe7c-9012-478c-b77f-f577fcd06120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:43.985450 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:44.003507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28fe7c-9012-478c-b77f-f577fcd06120 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:13:44.132168 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab05679-efe3-4a0e-bfbf-71db7cd9ffe4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:13:44.132806 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab05679-efe3-4a0e-bfbf-71db7cd9ffe4 service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:44.133086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab05679-efe3-4a0e-bfbf-71db7cd9ffe4 service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:44.133358 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab05679-efe3-4a0e-bfbf-71db7cd9ffe4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:44.134994 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-7ab05679-efe3-4a0e-bfbf-71db7cd9ffe4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:44.235469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab05679-efe3-4a0e-bfbf-71db7cd9ffe4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:13:44.237382 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab05679-efe3-4a0e-bfbf-71db7cd9ffe4 service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:13:44.248314 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:44.258386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 699/1382] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:13:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:13:46.561152 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6033d-e1d6-4da1-9cc7-9a4f86c85b79 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:46.561737 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6033d-e1d6-4da1-9cc7-9a4f86c85b79 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:46.561875 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6033d-e1d6-4da1-9cc7-9a4f86c85b79 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:46.564252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 684/1383] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:13:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:13:46.571136 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4fe7e6-1adc-4015-a6cd-a6b7d5cb681d None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:46.571650 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4fe7e6-1adc-4015-a6cd-a6b7d5cb681d None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:46.571830 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4fe7e6-1adc-4015-a6cd-a6b7d5cb681d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:46.660465 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-de4fe7e6-1adc-4015-a6cd-a6b7d5cb681d None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:13:46.662041 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-de4fe7e6-1adc-4015-a6cd-a6b7d5cb681d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:46.773307 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 700/1384] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:13:47.129194 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6033d-e1d6-4da1-9cc7-9a4f86c85b79 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:13:47.130737 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-aae6033d-e1d6-4da1-9cc7-9a4f86c85b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:47.296587 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6033d-e1d6-4da1-9cc7-9a4f86c85b79 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:13:47.379235 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:47.433060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c2111-a2fe-41da-b868-90e486620450 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:13:47.433888 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c2111-a2fe-41da-b868-90e486620450 service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:47.434236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c2111-a2fe-41da-b868-90e486620450 service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:47.434575 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c2111-a2fe-41da-b868-90e486620450 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:47.436625 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b92c2111-a2fe-41da-b868-90e486620450 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:47.532511 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c2111-a2fe-41da-b868-90e486620450 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:13:47.534113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c2111-a2fe-41da-b868-90e486620450 service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:13:47.552846 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 685/1385] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:13:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:13:49.803294 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f98a7-2f5d-448a-8e58-39baca9f1c71 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:49.803880 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f98a7-2f5d-448a-8e58-39baca9f1c71 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:49.803880 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f98a7-2f5d-448a-8e58-39baca9f1c71 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:49.805281 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 701/1386] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:13:49.811413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63edfaa3-9e9c-42c1-bf95-d2e77f351846 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:49.811910 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63edfaa3-9e9c-42c1-bf95-d2e77f351846 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:49.812016 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63edfaa3-9e9c-42c1-bf95-d2e77f351846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:49.889060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-63edfaa3-9e9c-42c1-bf95-d2e77f351846 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:13:49.890998 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-63edfaa3-9e9c-42c1-bf95-d2e77f351846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:49.934023 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:50.025466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 686/1387] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:13:50.427002 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f98a7-2f5d-448a-8e58-39baca9f1c71 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:13:50.428576 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a29f98a7-2f5d-448a-8e58-39baca9f1c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:50.513450 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:50.591882 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f98a7-2f5d-448a-8e58-39baca9f1c71 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:13:50.720841 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7bd84-f93c-44dd-b1ba-0fc2fa050a43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:13:50.721381 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7bd84-f93c-44dd-b1ba-0fc2fa050a43 service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:50.721662 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7bd84-f93c-44dd-b1ba-0fc2fa050a43 service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:50.721981 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7bd84-f93c-44dd-b1ba-0fc2fa050a43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:50.723774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-90d7bd84-f93c-44dd-b1ba-0fc2fa050a43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:50.765900 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:50.819065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d7bd84-f93c-44dd-b1ba-0fc2fa050a43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:13:50.820700 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d7bd84-f93c-44dd-b1ba-0fc2fa050a43 service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:13:50.840659 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 702/1388] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:13:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:13:52.730006 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b1c60-d001-496d-ac5e-d91e4a6c4bf7 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:52.730540 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b1c60-d001-496d-ac5e-d91e4a6c4bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:52.730540 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b1c60-d001-496d-ac5e-d91e4a6c4bf7 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:52.732044 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 687/1389] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:13:52.738189 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c71f8c-91d2-4202-b892-ce9fed719c3e None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:52.739454 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c71f8c-91d2-4202-b892-ce9fed719c3e None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:52.739960 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c71f8c-91d2-4202-b892-ce9fed719c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:52.803472 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f939a5-775f-45c1-af47-4c78e85279ec None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:52.803760 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f939a5-775f-45c1-af47-4c78e85279ec None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:52.804043 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f939a5-775f-45c1-af47-4c78e85279ec None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:52.805561 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 688/1390] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:13:52.810512 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31203381-7e23-43c1-aee0-fd862c0b29c9 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:52.810797 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31203381-7e23-43c1-aee0-fd862c0b29c9 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:52.811091 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31203381-7e23-43c1-aee0-fd862c0b29c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:52.814258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-70c71f8c-91d2-4202-b892-ce9fed719c3e None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:13:52.815538 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-70c71f8c-91d2-4202-b892-ce9fed719c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:52.881766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-31203381-7e23-43c1-aee0-fd862c0b29c9 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:13:52.882977 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-31203381-7e23-43c1-aee0-fd862c0b29c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:52.916213 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 703/1391] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:13:52.984578 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 689/1392] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:13:53.221521 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31c900-5f1c-459e-8b14-c8978a3cb841 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:53.221811 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31c900-5f1c-459e-8b14-c8978a3cb841 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:53.222084 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31c900-5f1c-459e-8b14-c8978a3cb841 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:53.223455 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 704/1393] 10.222.0.67 () {54 vars in 967 bytes} [Wed Aug 31 00:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:13:53.228441 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577236b-2492-486f-bd01-ed6e67c45ed5 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:53.228731 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577236b-2492-486f-bd01-ed6e67c45ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:53.229009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577236b-2492-486f-bd01-ed6e67c45ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:53.245020 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:53.275813 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa31c900-5f1c-459e-8b14-c8978a3cb841 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:13:53.277338 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-aa31c900-5f1c-459e-8b14-c8978a3cb841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:53.291758 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:53.308729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-5577236b-2492-486f-bd01-ed6e67c45ed5 None None] MFA Rules not processed for user `55714a21c1cb4f2b949603230243ed02`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:13:53.309925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-5577236b-2492-486f-bd01-ed6e67c45ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:53.389097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 690/1394] 10.222.0.67 () {58 vars in 1064 bytes} [Wed Aug 31 00:13:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:13:53.397863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577236b-2492-486f-bd01-ed6e67c45ed5 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:13:53.405958 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577236b-2492-486f-bd01-ed6e67c45ed5 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:13:53.440318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa31c900-5f1c-459e-8b14-c8978a3cb841 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:13:53.484138 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ebded-475d-4937-b9c4-d7d0404111a1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55714a21c1cb4f2b949603230243ed02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:13:53.484889 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ebded-475d-4937-b9c4-d7d0404111a1 service glance] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:53.485194 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ebded-475d-4937-b9c4-d7d0404111a1 service glance] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:53.485466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ebded-475d-4937-b9c4-d7d0404111a1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:53.487123 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-789ebded-475d-4937-b9c4-d7d0404111a1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:53.518405 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:53.530058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:53.559523 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f936d16-711c-44ac-99b5-ce449d72160d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:13:53.560141 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f936d16-711c-44ac-99b5-ce449d72160d service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:53.560424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f936d16-711c-44ac-99b5-ce449d72160d service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:53.560699 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f936d16-711c-44ac-99b5-ce449d72160d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:53.562827 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-6f936d16-711c-44ac-99b5-ce449d72160d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:53.610632 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ebded-475d-4937-b9c4-d7d0404111a1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:13:53.612111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ebded-475d-4937-b9c4-d7d0404111a1 service glance] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:13:53.632801 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 691/1395] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:13:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:13:53.658685 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f936d16-711c-44ac-99b5-ce449d72160d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:13:53.660336 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f936d16-711c-44ac-99b5-ce449d72160d service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:13:53.679367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 705/1396] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:13:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:13:54.415582 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ebb8a-99d1-43e8-9178-509b93aa2646 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:54.416118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ebb8a-99d1-43e8-9178-509b93aa2646 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:54.416118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ebb8a-99d1-43e8-9178-509b93aa2646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:54.505498 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-072ebb8a-99d1-43e8-9178-509b93aa2646 None None] MFA Rules not processed for user `a8fb7a9acefd4b418c992661037c2ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:13:54.506760 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-072ebb8a-99d1-43e8-9178-509b93aa2646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:54.597467 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:54.603904 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 692/1397] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 31 00:13:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:13:54.654480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb407c4-272e-4541-ba41-c6337fc60f7b None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:54.655009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb407c4-272e-4541-ba41-c6337fc60f7b None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:54.655707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb407c4-272e-4541-ba41-c6337fc60f7b None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:54.657128 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 706/1398] 10.222.0.67 () {54 vars in 943 bytes} [Wed Aug 31 00:13:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:13:54.663629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37db7ec-b5ef-49f7-8b29-1d40ab5a4dc6 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:54.663912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37db7ec-b5ef-49f7-8b29-1d40ab5a4dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:54.664216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37db7ec-b5ef-49f7-8b29-1d40ab5a4dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:54.747116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-a37db7ec-b5ef-49f7-8b29-1d40ab5a4dc6 None None] MFA Rules not processed for user `1cd3cfbc8ebd4c5a83bef09efe10bc91`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:13:54.748307 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-a37db7ec-b5ef-49f7-8b29-1d40ab5a4dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:54.863707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 693/1399] 10.222.0.67 () {58 vars in 1040 bytes} [Wed Aug 31 00:13:54 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:13:54.875000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb407c4-272e-4541-ba41-c6337fc60f7b None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:13:54.876623 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-bfb407c4-272e-4541-ba41-c6337fc60f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:54.976328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:55.021709 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb407c4-272e-4541-ba41-c6337fc60f7b None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:13:55.126119 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b308fd-f8d1-4ce7-a49a-7bbae5f75dd5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd3cfbc8ebd4c5a83bef09efe10bc91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:13:55.126647 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b308fd-f8d1-4ce7-a49a-7bbae5f75dd5 service swift] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:55.127105 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b308fd-f8d1-4ce7-a49a-7bbae5f75dd5 service swift] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:55.127494 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b308fd-f8d1-4ce7-a49a-7bbae5f75dd5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:55.129167 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-48b308fd-f8d1-4ce7-a49a-7bbae5f75dd5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:55.232371 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b308fd-f8d1-4ce7-a49a-7bbae5f75dd5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:13:55.234098 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b308fd-f8d1-4ce7-a49a-7bbae5f75dd5 service swift] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:13:55.237865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:55.244464 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 707/1400] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 31 00:13:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:13:55.718493 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca51d3-1c5c-4c51-ab2e-8de2e30a200e None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:55.719108 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca51d3-1c5c-4c51-ab2e-8de2e30a200e None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:55.719108 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca51d3-1c5c-4c51-ab2e-8de2e30a200e None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:55.724323 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 694/1401] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:13:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:13:55.725701 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7618a-0c83-4f9f-8673-8fb8e6e115b2 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:55.726115 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7618a-0c83-4f9f-8673-8fb8e6e115b2 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:55.726299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7618a-0c83-4f9f-8673-8fb8e6e115b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:55.796596 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-afd7618a-0c83-4f9f-8673-8fb8e6e115b2 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:13:55.797815 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-afd7618a-0c83-4f9f-8673-8fb8e6e115b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:55.901355 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 708/1402] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:13:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:13:56.193776 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca51d3-1c5c-4c51-ab2e-8de2e30a200e None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:13:56.196112 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-beca51d3-1c5c-4c51-ab2e-8de2e30a200e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:56.261256 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:56.366209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca51d3-1c5c-4c51-ab2e-8de2e30a200e None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:13:56.494970 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11259f-2e08-4c1d-86d6-71004ed42cd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:13:56.495615 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11259f-2e08-4c1d-86d6-71004ed42cd9 service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:56.495897 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11259f-2e08-4c1d-86d6-71004ed42cd9 service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:56.496169 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11259f-2e08-4c1d-86d6-71004ed42cd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:56.497787 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-2d11259f-2e08-4c1d-86d6-71004ed42cd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:56.518518 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:56.599161 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11259f-2e08-4c1d-86d6-71004ed42cd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:13:56.601044 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11259f-2e08-4c1d-86d6-71004ed42cd9 service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:13:56.622002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 695/1403] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:13:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:13:58.610370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7910a-f0b6-44e0-80c5-46a1dbf34b7b None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:58.611107 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7910a-f0b6-44e0-80c5-46a1dbf34b7b None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:58.611107 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7910a-f0b6-44e0-80c5-46a1dbf34b7b None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:58.612907 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 709/1404] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:13:58.618052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b581300-66d4-46cf-933b-34ad3ac4c3e3 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:58.618565 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b581300-66d4-46cf-933b-34ad3ac4c3e3 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:58.618673 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b581300-66d4-46cf-933b-34ad3ac4c3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:58.694397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-5b581300-66d4-46cf-933b-34ad3ac4c3e3 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:13:58.695921 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-5b581300-66d4-46cf-933b-34ad3ac4c3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:58.804976 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:58.817305 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 696/1405] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:13:59.160635 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba7910a-f0b6-44e0-80c5-46a1dbf34b7b None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:13:59.160895 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4ba7910a-f0b6-44e0-80c5-46a1dbf34b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:59.261081 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:59.338567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba7910a-f0b6-44e0-80c5-46a1dbf34b7b None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:13:59.467059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e2d44-cc25-471b-b986-a05d8b4b2f98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:13:59.467740 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e2d44-cc25-471b-b986-a05d8b4b2f98 service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:13:59.468026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e2d44-cc25-471b-b986-a05d8b4b2f98 service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:13:59.468292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e2d44-cc25-471b-b986-a05d8b4b2f98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:13:59.470078 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-172e2d44-cc25-471b-b986-a05d8b4b2f98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:13:59.516884 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:13:59.579741 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e2d44-cc25-471b-b986-a05d8b4b2f98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:13:59.581758 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e2d44-cc25-471b-b986-a05d8b4b2f98 service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:13:59.616615 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 710/1406] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:13:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:14:01.601478 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3dcb91-540d-4c42-82a5-fb629a7f5d69 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:01.602011 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3dcb91-540d-4c42-82a5-fb629a7f5d69 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:01.602011 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3dcb91-540d-4c42-82a5-fb629a7f5d69 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:01.603565 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 697/1407] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:01.609525 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4449b-9924-4db8-b2a6-e219a1ceb11f None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:01.609942 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4449b-9924-4db8-b2a6-e219a1ceb11f None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:01.610118 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4449b-9924-4db8-b2a6-e219a1ceb11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:01.694481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-7ca4449b-9924-4db8-b2a6-e219a1ceb11f None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:01.695749 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-7ca4449b-9924-4db8-b2a6-e219a1ceb11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:01.810634 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 711/1408] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:14:02.103421 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3dcb91-540d-4c42-82a5-fb629a7f5d69 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:02.104982 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-cb3dcb91-540d-4c42-82a5-fb629a7f5d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:02.283995 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3dcb91-540d-4c42-82a5-fb629a7f5d69 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:02.356538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:02.416339 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941aa57-4e1f-49aa-965e-599fe8854ee7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:02.417088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941aa57-4e1f-49aa-965e-599fe8854ee7 service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:02.417376 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941aa57-4e1f-49aa-965e-599fe8854ee7 service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:02.417648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941aa57-4e1f-49aa-965e-599fe8854ee7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:02.419334 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f941aa57-4e1f-49aa-965e-599fe8854ee7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:02.528747 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f941aa57-4e1f-49aa-965e-599fe8854ee7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:02.530570 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f941aa57-4e1f-49aa-965e-599fe8854ee7 service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:02.558227 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 698/1409] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:14:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:14:11.807228 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=93 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:12.550590 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=65 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:17.706900 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24653be-ee2b-426c-9fd7-148b6db312cb None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:17.707471 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24653be-ee2b-426c-9fd7-148b6db312cb None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:17.707471 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24653be-ee2b-426c-9fd7-148b6db312cb None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:17.708870 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 712/1410] 10.222.0.67 () {54 vars in 918 bytes} [Wed Aug 31 00:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:17.713870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfae53-1ef8-4782-ab64-191bc3241127 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:17.714314 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfae53-1ef8-4782-ab64-191bc3241127 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:17.714422 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfae53-1ef8-4782-ab64-191bc3241127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:17.794240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-f0cfae53-1ef8-4782-ab64-191bc3241127 None None] MFA Rules not processed for user `fdacdcb2451742938164fdd44cdbaebe`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:17.795992 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f0cfae53-1ef8-4782-ab64-191bc3241127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:17.899362 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 699/1411] 10.222.0.67 () {58 vars in 1015 bytes} [Wed Aug 31 00:14:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:14:17.913590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001137e-f8c5-4c5d-aeda-368d5f48ada9 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:17.913875 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001137e-f8c5-4c5d-aeda-368d5f48ada9 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:17.914152 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001137e-f8c5-4c5d-aeda-368d5f48ada9 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:17.915543 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 713/1412] 10.222.0.67 () {54 vars in 918 bytes} [Wed Aug 31 00:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:17.920864 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5a384-be2d-4903-8ba9-dd1d7b53e89b None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:17.921318 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5a384-be2d-4903-8ba9-dd1d7b53e89b None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:17.921597 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5a384-be2d-4903-8ba9-dd1d7b53e89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:17.994465 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:17.997644 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-b9c5a384-be2d-4903-8ba9-dd1d7b53e89b None None] MFA Rules not processed for user `3aa53ae5a30146f69d7d6eefe4290dba`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:17.999779 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b9c5a384-be2d-4903-8ba9-dd1d7b53e89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:18.126804 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 700/1413] 10.222.0.67 () {58 vars in 1015 bytes} [Wed Aug 31 00:14:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:14:20.542662 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124bfb6-8885-416f-b848-218a0be55d7f None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:20.543258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124bfb6-8885-416f-b848-218a0be55d7f None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:20.543258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124bfb6-8885-416f-b848-218a0be55d7f None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:20.544596 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 714/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:14:20.550571 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c317a5e-a0d9-4cf7-9835-c8ccff0c6231 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:20.551009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c317a5e-a0d9-4cf7-9835-c8ccff0c6231 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:20.551149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c317a5e-a0d9-4cf7-9835-c8ccff0c6231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:20.617160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-2c317a5e-a0d9-4cf7-9835-c8ccff0c6231 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:20.618374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-2c317a5e-a0d9-4cf7-9835-c8ccff0c6231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:20.674922 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:20.718019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 701/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:14:21.011368 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124bfb6-8885-416f-b848-218a0be55d7f None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:21.012912 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-d124bfb6-8885-416f-b848-218a0be55d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:21.158701 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124bfb6-8885-416f-b848-218a0be55d7f None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:21.229431 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:21.269921 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b02a2a-68e2-4177-a2ea-bda4f649ef01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:21.270450 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b02a2a-68e2-4177-a2ea-bda4f649ef01 service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:21.270724 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b02a2a-68e2-4177-a2ea-bda4f649ef01 service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:21.270992 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b02a2a-68e2-4177-a2ea-bda4f649ef01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:21.272612 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-c7b02a2a-68e2-4177-a2ea-bda4f649ef01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:21.360524 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b02a2a-68e2-4177-a2ea-bda4f649ef01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:21.362132 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b02a2a-68e2-4177-a2ea-bda4f649ef01 service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:21.380927 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 715/1416] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:14:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:14:22.280465 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4997b-332c-4899-971e-a3dcd0dff360 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:22.281021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4997b-332c-4899-971e-a3dcd0dff360 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:22.281021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4997b-332c-4899-971e-a3dcd0dff360 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:22.282311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 702/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 00:14:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:22.287488 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9067a-7031-41c7-9d3b-34732106db7f None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:22.287913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9067a-7031-41c7-9d3b-34732106db7f None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:22.288034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9067a-7031-41c7-9d3b-34732106db7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:22.362830 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-07c9067a-7031-41c7-9d3b-34732106db7f None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:22.364059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-07c9067a-7031-41c7-9d3b-34732106db7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:22.391521 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:22.461268 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 716/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 00:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:14:22.638343 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4997b-332c-4899-971e-a3dcd0dff360 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:22.639941 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-53b4997b-332c-4899-971e-a3dcd0dff360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:22.791374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4997b-332c-4899-971e-a3dcd0dff360 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:22.832030 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:22.920693 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e3230-714c-4826-b155-bc377cad7881 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:22.921530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e3230-714c-4826-b155-bc377cad7881 service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:22.921956 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e3230-714c-4826-b155-bc377cad7881 service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:22.922373 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e3230-714c-4826-b155-bc377cad7881 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:22.924801 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-d79e3230-714c-4826-b155-bc377cad7881 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:23.027249 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e3230-714c-4826-b155-bc377cad7881 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:23.028929 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e3230-714c-4826-b155-bc377cad7881 service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:23.048159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 703/1419] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:14:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:14:23.953313 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb32681-9f9c-4aef-b2dc-90109e64c4b1 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:23.953890 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb32681-9f9c-4aef-b2dc-90109e64c4b1 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:23.953890 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb32681-9f9c-4aef-b2dc-90109e64c4b1 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:23.955255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 717/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 00:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:23.960331 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898e50d-4633-4549-8c46-677fd4db6b77 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:23.960769 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898e50d-4633-4549-8c46-677fd4db6b77 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:23.960908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898e50d-4633-4549-8c46-677fd4db6b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:24.018779 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:24.040651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-5898e50d-4633-4549-8c46-677fd4db6b77 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:24.041858 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-5898e50d-4633-4549-8c46-677fd4db6b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:24.139818 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 704/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 00:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:14:24.290511 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb32681-9f9c-4aef-b2dc-90109e64c4b1 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:24.292009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-5cb32681-9f9c-4aef-b2dc-90109e64c4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:24.439003 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb32681-9f9c-4aef-b2dc-90109e64c4b1 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:24.450962 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:24.562030 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e37e1-2612-413e-a555-a60239e04b37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:24.562559 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e37e1-2612-413e-a555-a60239e04b37 service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:24.562842 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e37e1-2612-413e-a555-a60239e04b37 service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:24.563218 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e37e1-2612-413e-a555-a60239e04b37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:24.564847 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-fc9e37e1-2612-413e-a555-a60239e04b37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:24.653438 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9e37e1-2612-413e-a555-a60239e04b37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:24.655413 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9e37e1-2612-413e-a555-a60239e04b37 service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:24.673901 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 718/1422] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:14:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:14:25.564269 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf01d3-e348-4909-9747-9b3a3d4300c6 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:25.564895 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf01d3-e348-4909-9747-9b3a3d4300c6 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:25.564895 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf01d3-e348-4909-9747-9b3a3d4300c6 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:25.566257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 705/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 00:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:25.576006 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0ddcb-a272-4039-9d4b-b7f7827fef01 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:25.576739 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0ddcb-a272-4039-9d4b-b7f7827fef01 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:25.576739 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0ddcb-a272-4039-9d4b-b7f7827fef01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:25.599788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:25.657490 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-44d0ddcb-a272-4039-9d4b-b7f7827fef01 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:25.658736 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-44d0ddcb-a272-4039-9d4b-b7f7827fef01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:25.756847 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 719/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 00:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:14:25.910205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf01d3-e348-4909-9747-9b3a3d4300c6 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:25.912105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-40bf01d3-e348-4909-9747-9b3a3d4300c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:26.035660 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:26.071419 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf01d3-e348-4909-9747-9b3a3d4300c6 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:26.183350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45582eca-9dad-4702-9386-cba74c5aa31f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:26.183870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45582eca-9dad-4702-9386-cba74c5aa31f service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:26.184145 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45582eca-9dad-4702-9386-cba74c5aa31f service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:26.184412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45582eca-9dad-4702-9386-cba74c5aa31f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:26.186332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-45582eca-9dad-4702-9386-cba74c5aa31f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:26.277266 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:26.295414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45582eca-9dad-4702-9386-cba74c5aa31f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:26.297077 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45582eca-9dad-4702-9386-cba74c5aa31f service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:26.317341 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 706/1425] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:14:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:14:27.192866 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b26482-c97c-442f-a18e-297b06fa5948 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:27.193460 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b26482-c97c-442f-a18e-297b06fa5948 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:27.193460 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b26482-c97c-442f-a18e-297b06fa5948 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:27.194763 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 720/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 00:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:27.199863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443d996-b213-4571-9f67-655a14ff4e13 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:27.200354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443d996-b213-4571-9f67-655a14ff4e13 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:27.200573 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443d996-b213-4571-9f67-655a14ff4e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:27.276903 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-4443d996-b213-4571-9f67-655a14ff4e13 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:27.278059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-4443d996-b213-4571-9f67-655a14ff4e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:27.376558 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 707/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 00:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:14:27.527964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b26482-c97c-442f-a18e-297b06fa5948 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:27.530455 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e2b26482-c97c-442f-a18e-297b06fa5948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:27.614828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:27.684223 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b26482-c97c-442f-a18e-297b06fa5948 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:27.798988 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad7a13-666f-4da9-9a77-a6793ea0efc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:27.799552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad7a13-666f-4da9-9a77-a6793ea0efc1 service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:27.799832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad7a13-666f-4da9-9a77-a6793ea0efc1 service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:27.800106 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad7a13-666f-4da9-9a77-a6793ea0efc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:27.801723 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-49ad7a13-666f-4da9-9a77-a6793ea0efc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:27.850467 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:27.893021 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad7a13-666f-4da9-9a77-a6793ea0efc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:27.894640 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad7a13-666f-4da9-9a77-a6793ea0efc1 service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:27.912738 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 721/1428] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:14:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:14:28.716401 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db757010-dea1-4f29-ac78-3fc4ba49a731 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:28.716930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db757010-dea1-4f29-ac78-3fc4ba49a731 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:28.716930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db757010-dea1-4f29-ac78-3fc4ba49a731 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:28.718281 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 708/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 00:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:28.723324 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f06ca-798a-4757-8856-b70bdc9737ab None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:28.723794 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f06ca-798a-4757-8856-b70bdc9737ab None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:28.723908 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f06ca-798a-4757-8856-b70bdc9737ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:28.801403 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-d08f06ca-798a-4757-8856-b70bdc9737ab None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:28.802514 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-d08f06ca-798a-4757-8856-b70bdc9737ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:28.902736 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 722/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 00:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:14:29.059092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db757010-dea1-4f29-ac78-3fc4ba49a731 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:29.060692 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-db757010-dea1-4f29-ac78-3fc4ba49a731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:29.108858 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:29.206746 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db757010-dea1-4f29-ac78-3fc4ba49a731 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:29.319514 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2c042-a7bd-4e9a-8d2e-216dbacb12d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:29.320042 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2c042-a7bd-4e9a-8d2e-216dbacb12d2 service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:29.320325 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2c042-a7bd-4e9a-8d2e-216dbacb12d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:29.320593 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2c042-a7bd-4e9a-8d2e-216dbacb12d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:29.322195 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-aac2c042-a7bd-4e9a-8d2e-216dbacb12d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:29.329707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:29.413236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2c042-a7bd-4e9a-8d2e-216dbacb12d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:29.414828 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2c042-a7bd-4e9a-8d2e-216dbacb12d2 service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:29.432379 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 709/1431] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:14:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:14:30.235663 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910f464-02c3-4acc-a2fa-267ef429fb23 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:30.236256 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910f464-02c3-4acc-a2fa-267ef429fb23 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:30.236256 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910f464-02c3-4acc-a2fa-267ef429fb23 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:30.237538 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 723/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 00:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:30.242487 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a81a1-79e4-4d5f-ab2e-3c866c35977b None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:30.242926 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a81a1-79e4-4d5f-ab2e-3c866c35977b None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:30.243063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a81a1-79e4-4d5f-ab2e-3c866c35977b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:30.320003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-e63a81a1-79e4-4d5f-ab2e-3c866c35977b None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:30.321114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-e63a81a1-79e4-4d5f-ab2e-3c866c35977b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:30.389281 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:30.418195 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 710/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 00:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:14:30.568565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910f464-02c3-4acc-a2fa-267ef429fb23 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:30.570151 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a910f464-02c3-4acc-a2fa-267ef429fb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:30.588111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:30.715338 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910f464-02c3-4acc-a2fa-267ef429fb23 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:30.801047 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:30.827390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e56c9bb-2332-45b3-8d9b-737f2770a298 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:30.827992 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56c9bb-2332-45b3-8d9b-737f2770a298 service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:30.828284 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56c9bb-2332-45b3-8d9b-737f2770a298 service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:30.828541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56c9bb-2332-45b3-8d9b-737f2770a298 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:30.830143 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4e56c9bb-2332-45b3-8d9b-737f2770a298 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:30.923439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e56c9bb-2332-45b3-8d9b-737f2770a298 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:30.925057 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e56c9bb-2332-45b3-8d9b-737f2770a298 service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:30.943938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 724/1434] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:14:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:14:31.752715 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883c80c-a5ec-4098-b461-755383d60857 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:31.753294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883c80c-a5ec-4098-b461-755383d60857 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:31.753294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883c80c-a5ec-4098-b461-755383d60857 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:31.754658 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 711/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 00:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:31.759652 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da380bb-e5b9-4ba9-ac53-e28b61546c9e None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:31.760067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da380bb-e5b9-4ba9-ac53-e28b61546c9e None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:31.760214 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da380bb-e5b9-4ba9-ac53-e28b61546c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:31.838145 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-4da380bb-e5b9-4ba9-ac53-e28b61546c9e None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:31.839467 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4da380bb-e5b9-4ba9-ac53-e28b61546c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:31.881851 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:31.937582 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 725/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 00:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:14:32.088013 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883c80c-a5ec-4098-b461-755383d60857 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:32.089547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-e883c80c-a5ec-4098-b461-755383d60857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:32.236723 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883c80c-a5ec-4098-b461-755383d60857 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:32.290449 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:32.349636 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6daea0a-7a94-485c-8db9-43c7de54b864 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:32.350168 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6daea0a-7a94-485c-8db9-43c7de54b864 service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:32.350444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6daea0a-7a94-485c-8db9-43c7de54b864 service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:32.350713 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6daea0a-7a94-485c-8db9-43c7de54b864 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:32.352333 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c6daea0a-7a94-485c-8db9-43c7de54b864 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:32.446582 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6daea0a-7a94-485c-8db9-43c7de54b864 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:32.448203 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6daea0a-7a94-485c-8db9-43c7de54b864 service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:32.467304 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 712/1437] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:14:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:14:33.286893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a1028-8096-40fb-b0bb-b5ad3ddf156a None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:33.287474 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a1028-8096-40fb-b0bb-b5ad3ddf156a None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:33.287588 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a1028-8096-40fb-b0bb-b5ad3ddf156a None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:33.288976 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 726/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 00:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:33.294139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a083792-c3e1-4120-a386-bab55b0005f8 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:33.294566 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a083792-c3e1-4120-a386-bab55b0005f8 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:33.294670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a083792-c3e1-4120-a386-bab55b0005f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:33.370904 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-1a083792-c3e1-4120-a386-bab55b0005f8 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:33.372790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-1a083792-c3e1-4120-a386-bab55b0005f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:33.374880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:33.469192 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 713/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 00:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:14:33.627309 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a1028-8096-40fb-b0bb-b5ad3ddf156a None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:33.628879 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4f2a1028-8096-40fb-b0bb-b5ad3ddf156a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:33.776414 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a1028-8096-40fb-b0bb-b5ad3ddf156a None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:33.803107 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:33.890212 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14344da1-3395-4ab3-b3ae-4e28a0819a7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:33.890824 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14344da1-3395-4ab3-b3ae-4e28a0819a7c service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:33.891131 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14344da1-3395-4ab3-b3ae-4e28a0819a7c service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:33.891406 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14344da1-3395-4ab3-b3ae-4e28a0819a7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:33.893304 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-14344da1-3395-4ab3-b3ae-4e28a0819a7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:33.985205 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14344da1-3395-4ab3-b3ae-4e28a0819a7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:33.986851 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14344da1-3395-4ab3-b3ae-4e28a0819a7c service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:34.005448 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 727/1440] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:14:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:14:35.843954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146deb2-71ae-44d8-8359-9e973318e5a1 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:35.844524 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146deb2-71ae-44d8-8359-9e973318e5a1 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:35.844524 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146deb2-71ae-44d8-8359-9e973318e5a1 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:35.845829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 714/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:35.851002 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d01a7-bec1-4f3b-b6cf-9be57cb23781 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:35.851424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d01a7-bec1-4f3b-b6cf-9be57cb23781 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:35.851583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d01a7-bec1-4f3b-b6cf-9be57cb23781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:35.893799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:35.920395 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-e06d01a7-bec1-4f3b-b6cf-9be57cb23781 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:35.921602 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e06d01a7-bec1-4f3b-b6cf-9be57cb23781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:36.019148 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 728/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:14:36.300985 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146deb2-71ae-44d8-8359-9e973318e5a1 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:36.302530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-4146deb2-71ae-44d8-8359-9e973318e5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:36.406867 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146deb2-71ae-44d8-8359-9e973318e5a1 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:36.437995 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:36.479900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ee419-7c05-4473-b3a5-d1e5dbe92d4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:36.480859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ee419-7c05-4473-b3a5-d1e5dbe92d4e service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:36.480859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ee419-7c05-4473-b3a5-d1e5dbe92d4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:36.481154 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ee419-7c05-4473-b3a5-d1e5dbe92d4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:36.482915 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-d12ee419-7c05-4473-b3a5-d1e5dbe92d4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:36.588716 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ee419-7c05-4473-b3a5-d1e5dbe92d4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:36.590721 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ee419-7c05-4473-b3a5-d1e5dbe92d4e service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:36.610171 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 715/1443] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:14:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:14:38.692277 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005ef2b-99ef-4929-8983-159262df0315 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:38.692834 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005ef2b-99ef-4929-8983-159262df0315 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:38.692834 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005ef2b-99ef-4929-8983-159262df0315 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:38.694162 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 729/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:38.699684 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15eb3e0-dbae-4a75-b5fc-6df379225a70 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:38.700109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15eb3e0-dbae-4a75-b5fc-6df379225a70 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:38.700263 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15eb3e0-dbae-4a75-b5fc-6df379225a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:38.767301 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-b15eb3e0-dbae-4a75-b5fc-6df379225a70 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:38.768434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b15eb3e0-dbae-4a75-b5fc-6df379225a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:38.781192 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:38.865558 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 716/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:14:39.159620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005ef2b-99ef-4929-8983-159262df0315 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:39.161207 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-9005ef2b-99ef-4929-8983-159262df0315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:39.263205 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005ef2b-99ef-4929-8983-159262df0315 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:39.275306 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:39.336734 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dcb159-da59-42b2-af37-99f48c0dc685 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:39.337336 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcb159-da59-42b2-af37-99f48c0dc685 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:39.337614 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcb159-da59-42b2-af37-99f48c0dc685 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:39.337905 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcb159-da59-42b2-af37-99f48c0dc685 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:39.339563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e2dcb159-da59-42b2-af37-99f48c0dc685 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:39.447446 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dcb159-da59-42b2-af37-99f48c0dc685 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:39.448883 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dcb159-da59-42b2-af37-99f48c0dc685 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:39.466620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 730/1446] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:14:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:14:41.797743 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a690778-347b-47a7-82f3-c2b6821e04b6 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:41.798290 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a690778-347b-47a7-82f3-c2b6821e04b6 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:41.798501 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a690778-347b-47a7-82f3-c2b6821e04b6 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:41.799906 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 717/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:41.805501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80867afa-d20b-4aa5-bebf-9ad595f9847b None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:41.805980 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80867afa-d20b-4aa5-bebf-9ad595f9847b None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:41.807146 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80867afa-d20b-4aa5-bebf-9ad595f9847b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:41.864438 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:41.883180 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-80867afa-d20b-4aa5-bebf-9ad595f9847b None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:41.884425 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-80867afa-d20b-4aa5-bebf-9ad595f9847b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:41.980539 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 731/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:14:42.272564 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a690778-347b-47a7-82f3-c2b6821e04b6 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:42.274104 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-6a690778-347b-47a7-82f3-c2b6821e04b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:42.379177 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a690778-347b-47a7-82f3-c2b6821e04b6 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:42.420549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:42.455004 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f4a84-2a5d-4ff1-97dd-5e853edc7124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:42.455462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f4a84-2a5d-4ff1-97dd-5e853edc7124 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:42.455741 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f4a84-2a5d-4ff1-97dd-5e853edc7124 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:42.456017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f4a84-2a5d-4ff1-97dd-5e853edc7124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:42.457615 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f14f4a84-2a5d-4ff1-97dd-5e853edc7124 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:42.567537 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f4a84-2a5d-4ff1-97dd-5e853edc7124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:42.568902 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f4a84-2a5d-4ff1-97dd-5e853edc7124 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:42.587352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 718/1449] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:14:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:14:44.686371 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5931a02f-531a-49ec-a69a-a720e6b13018 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:44.686895 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5931a02f-531a-49ec-a69a-a720e6b13018 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:44.687013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5931a02f-531a-49ec-a69a-a720e6b13018 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:44.688298 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 732/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:44.693666 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba557b0a-f279-4376-b74f-d22682e19fd7 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:44.694115 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba557b0a-f279-4376-b74f-d22682e19fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:44.694222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba557b0a-f279-4376-b74f-d22682e19fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:44.760106 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-ba557b0a-f279-4376-b74f-d22682e19fd7 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:44.761308 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-ba557b0a-f279-4376-b74f-d22682e19fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:44.781851 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:44.858083 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 719/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:14:45.142982 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5931a02f-531a-49ec-a69a-a720e6b13018 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:45.144556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-5931a02f-531a-49ec-a69a-a720e6b13018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:45.248568 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5931a02f-531a-49ec-a69a-a720e6b13018 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:45.268494 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:45.322450 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a15886-4052-43cd-a615-637966f65030 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:45.323075 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a15886-4052-43cd-a615-637966f65030 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:45.323473 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a15886-4052-43cd-a615-637966f65030 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:45.323779 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a15886-4052-43cd-a615-637966f65030 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:45.325412 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-39a15886-4052-43cd-a615-637966f65030 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:45.436964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a15886-4052-43cd-a615-637966f65030 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:45.438329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a15886-4052-43cd-a615-637966f65030 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:45.456570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 733/1452] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:14:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:14:47.438110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914e491-c451-4c67-9d60-284bb5746467 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:47.438649 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914e491-c451-4c67-9d60-284bb5746467 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:47.438767 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914e491-c451-4c67-9d60-284bb5746467 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:47.440228 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 720/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:47.445462 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6d19d-07ac-44aa-98a6-af5f7df5d7ad None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:47.445884 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6d19d-07ac-44aa-98a6-af5f7df5d7ad None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:47.445997 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6d19d-07ac-44aa-98a6-af5f7df5d7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:47.509353 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:47.512883 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-5ce6d19d-07ac-44aa-98a6-af5f7df5d7ad None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:47.514685 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-5ce6d19d-07ac-44aa-98a6-af5f7df5d7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:47.612268 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 734/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:14:47.898172 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914e491-c451-4c67-9d60-284bb5746467 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:47.899788 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f914e491-c451-4c67-9d60-284bb5746467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:48.004349 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914e491-c451-4c67-9d60-284bb5746467 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:48.052517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:48.078100 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f5141-b239-4b7e-b8c7-cb65edfd3865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:48.078710 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f5141-b239-4b7e-b8c7-cb65edfd3865 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:48.078990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f5141-b239-4b7e-b8c7-cb65edfd3865 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:48.079290 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f5141-b239-4b7e-b8c7-cb65edfd3865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:48.080919 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-215f5141-b239-4b7e-b8c7-cb65edfd3865 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:48.190104 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f5141-b239-4b7e-b8c7-cb65edfd3865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:48.191481 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f5141-b239-4b7e-b8c7-cb65edfd3865 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:48.210093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 721/1455] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:14:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:14:50.415842 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f5eb0-3528-4de2-9fe8-9a942850ecd4 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:50.416372 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f5eb0-3528-4de2-9fe8-9a942850ecd4 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:50.416372 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f5eb0-3528-4de2-9fe8-9a942850ecd4 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:50.417795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 735/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:50.423153 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04011e56-0da8-459f-9e33-726738410de6 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:50.423595 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04011e56-0da8-459f-9e33-726738410de6 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:50.423720 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04011e56-0da8-459f-9e33-726738410de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:50.490856 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-04011e56-0da8-459f-9e33-726738410de6 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:50.492102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-04011e56-0da8-459f-9e33-726738410de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:50.522933 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:50.592126 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 722/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:14:50.877867 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f5eb0-3528-4de2-9fe8-9a942850ecd4 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:50.879480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-ec0f5eb0-3528-4de2-9fe8-9a942850ecd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:50.986103 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f5eb0-3528-4de2-9fe8-9a942850ecd4 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:51.016856 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:51.063235 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b042b0-16c4-4515-a959-cb28e429c745 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:51.063932 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b042b0-16c4-4515-a959-cb28e429c745 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:51.064215 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b042b0-16c4-4515-a959-cb28e429c745 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:51.064489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b042b0-16c4-4515-a959-cb28e429c745 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:51.066106 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-39b042b0-16c4-4515-a959-cb28e429c745 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:51.174620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b042b0-16c4-4515-a959-cb28e429c745 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:51.176049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b042b0-16c4-4515-a959-cb28e429c745 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:51.194910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 736/1458] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:14:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:14:53.278280 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca97403-6ff3-42d6-a03e-9ab5864e1ce1 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:53.278813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca97403-6ff3-42d6-a03e-9ab5864e1ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:53.278813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca97403-6ff3-42d6-a03e-9ab5864e1ce1 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:53.280132 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 723/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:53.285430 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac33c2b-d4ac-49a4-9b01-3b68884b5d83 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:53.285844 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac33c2b-d4ac-49a4-9b01-3b68884b5d83 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:53.286001 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac33c2b-d4ac-49a4-9b01-3b68884b5d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:53.353546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-4ac33c2b-d4ac-49a4-9b01-3b68884b5d83 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:53.355632 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4ac33c2b-d4ac-49a4-9b01-3b68884b5d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:53.363909 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:53.452851 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 737/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:14:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:14:53.736271 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca97403-6ff3-42d6-a03e-9ab5864e1ce1 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:53.737824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-4ca97403-6ff3-42d6-a03e-9ab5864e1ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:53.958546 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca97403-6ff3-42d6-a03e-9ab5864e1ce1 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:54.020473 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:54.038746 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ffb9f-73ee-4f3b-9d5f-2a148e219d97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:54.039376 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ffb9f-73ee-4f3b-9d5f-2a148e219d97 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:54.039653 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ffb9f-73ee-4f3b-9d5f-2a148e219d97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:54.039923 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ffb9f-73ee-4f3b-9d5f-2a148e219d97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:54.041599 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b12ffb9f-73ee-4f3b-9d5f-2a148e219d97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:54.147994 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ffb9f-73ee-4f3b-9d5f-2a148e219d97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:54.149395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ffb9f-73ee-4f3b-9d5f-2a148e219d97 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:54.167024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 724/1461] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:14:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:14:56.181522 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e40fba-be0c-477d-b8f3-3a499d142625 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:56.182361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e40fba-be0c-477d-b8f3-3a499d142625 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:56.182361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e40fba-be0c-477d-b8f3-3a499d142625 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:56.183807 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 738/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:56.188964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac128af-e48d-497f-a2a6-bec5afd4f449 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:56.189390 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac128af-e48d-497f-a2a6-bec5afd4f449 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:56.189518 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac128af-e48d-497f-a2a6-bec5afd4f449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:56.256869 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-1ac128af-e48d-497f-a2a6-bec5afd4f449 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:56.257979 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-1ac128af-e48d-497f-a2a6-bec5afd4f449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:56.299634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:56.377851 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 725/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:14:56.665284 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e40fba-be0c-477d-b8f3-3a499d142625 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:56.668666 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-d7e40fba-be0c-477d-b8f3-3a499d142625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:56.780985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e40fba-be0c-477d-b8f3-3a499d142625 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:56.817715 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:56.854784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f412bb-129f-4d4c-ba18-5809d7872834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:56.855419 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f412bb-129f-4d4c-ba18-5809d7872834 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:56.855811 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f412bb-129f-4d4c-ba18-5809d7872834 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:56.856086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f412bb-129f-4d4c-ba18-5809d7872834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:56.857764 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a3f412bb-129f-4d4c-ba18-5809d7872834 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:56.970028 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f412bb-129f-4d4c-ba18-5809d7872834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:56.971448 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f412bb-129f-4d4c-ba18-5809d7872834 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:56.990350 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 739/1464] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:14:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:14:58.965980 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3b4cb-5457-42e4-b60d-54b202bdc345 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:58.966551 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3b4cb-5457-42e4-b60d-54b202bdc345 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:58.966551 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3b4cb-5457-42e4-b60d-54b202bdc345 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:58.967900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 726/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:14:58.973412 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ca21b-4a46-4a1b-ab0a-f16c29c62461 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:58.973852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ca21b-4a46-4a1b-ab0a-f16c29c62461 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:58.973965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ca21b-4a46-4a1b-ab0a-f16c29c62461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:59.046598 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-998ca21b-4a46-4a1b-ab0a-f16c29c62461 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:14:59.047861 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-998ca21b-4a46-4a1b-ab0a-f16c29c62461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:59.066507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:59.146903 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 740/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:14:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:14:59.427760 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3b4cb-5457-42e4-b60d-54b202bdc345 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:14:59.429306 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-33e3b4cb-5457-42e4-b60d-54b202bdc345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:59.533856 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3b4cb-5457-42e4-b60d-54b202bdc345 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:14:59.604131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:14:59.611953 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca3faaa-c80d-427c-8374-89525c1fb2e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:14:59.612563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3faaa-c80d-427c-8374-89525c1fb2e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:14:59.612847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3faaa-c80d-427c-8374-89525c1fb2e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:14:59.613113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3faaa-c80d-427c-8374-89525c1fb2e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:14:59.614726 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-dca3faaa-c80d-427c-8374-89525c1fb2e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:14:59.725650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca3faaa-c80d-427c-8374-89525c1fb2e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:14:59.726997 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca3faaa-c80d-427c-8374-89525c1fb2e8 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:14:59.744996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 727/1467] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:14:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:15:01.670568 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354e462-8f2d-4104-b887-acdb7978e715 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:01.671158 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354e462-8f2d-4104-b887-acdb7978e715 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:01.671158 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354e462-8f2d-4104-b887-acdb7978e715 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:01.672602 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 741/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:01.677901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f07ec1-cba3-473d-be51-1be34dfbe395 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:01.678344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f07ec1-cba3-473d-be51-1be34dfbe395 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:01.678455 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f07ec1-cba3-473d-be51-1be34dfbe395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:01.746071 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-66f07ec1-cba3-473d-be51-1be34dfbe395 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:01.747313 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-66f07ec1-cba3-473d-be51-1be34dfbe395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:01.796769 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:01.845495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 728/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:15:02.137901 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354e462-8f2d-4104-b887-acdb7978e715 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:02.139768 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-5354e462-8f2d-4104-b887-acdb7978e715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:02.244789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354e462-8f2d-4104-b887-acdb7978e715 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:02.292740 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:02.318914 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b1f960-fd52-4f45-b6f2-6c06ad4b85c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:02.319544 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1f960-fd52-4f45-b6f2-6c06ad4b85c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:02.319824 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1f960-fd52-4f45-b6f2-6c06ad4b85c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:02.320095 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1f960-fd52-4f45-b6f2-6c06ad4b85c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:02.321723 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-d7b1f960-fd52-4f45-b6f2-6c06ad4b85c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:02.430588 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b1f960-fd52-4f45-b6f2-6c06ad4b85c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:02.432866 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b1f960-fd52-4f45-b6f2-6c06ad4b85c6 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:02.452419 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 742/1470] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:15:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:15:11.843715 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=22 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:12.450005 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=66 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:22.125916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a6649-cacf-40ba-ab96-d58f7b8489ae None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:22.126477 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a6649-cacf-40ba-ab96-d58f7b8489ae None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:22.126477 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a6649-cacf-40ba-ab96-d58f7b8489ae None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:22.127859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 729/1471] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:22.132953 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c591552-56a3-4918-bd8d-cae334bdd676 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:22.133349 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c591552-56a3-4918-bd8d-cae334bdd676 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:22.133505 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c591552-56a3-4918-bd8d-cae334bdd676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:22.199674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-3c591552-56a3-4918-bd8d-cae334bdd676 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:22.200861 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-3c591552-56a3-4918-bd8d-cae334bdd676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:22.296482 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 743/1472] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:15:22.593526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043e8c8-bfd4-4f6b-95bb-bcd08246f3a5 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:22.593833 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043e8c8-bfd4-4f6b-95bb-bcd08246f3a5 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:22.594089 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043e8c8-bfd4-4f6b-95bb-bcd08246f3a5 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:22.595461 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 730/1473] 10.222.0.67 () {54 vars in 967 bytes} [Wed Aug 31 00:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:22.600201 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18a348-cab1-4d59-acc8-2618f80d2418 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:22.600491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18a348-cab1-4d59-acc8-2618f80d2418 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:22.600767 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18a348-cab1-4d59-acc8-2618f80d2418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:22.677273 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-cb18a348-cab1-4d59-acc8-2618f80d2418 None None] MFA Rules not processed for user `55714a21c1cb4f2b949603230243ed02`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:22.678473 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-cb18a348-cab1-4d59-acc8-2618f80d2418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:22.699211 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:22.754850 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 744/1474] 10.222.0.67 () {58 vars in 1064 bytes} [Wed Aug 31 00:15:22 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:15:22.762254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043e8c8-bfd4-4f6b-95bb-bcd08246f3a5 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:22.764073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-7043e8c8-bfd4-4f6b-95bb-bcd08246f3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:22.867174 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043e8c8-bfd4-4f6b-95bb-bcd08246f3a5 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:22.940025 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1817a8-4c1b-4937-a904-936a66dbed44 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55714a21c1cb4f2b949603230243ed02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:22.940626 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1817a8-4c1b-4937-a904-936a66dbed44 service glance] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:22.940903 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1817a8-4c1b-4937-a904-936a66dbed44 service glance] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:22.941172 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1817a8-4c1b-4937-a904-936a66dbed44 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:22.942756 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-9c1817a8-4c1b-4937-a904-936a66dbed44 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:22.998178 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:23.052937 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1817a8-4c1b-4937-a904-936a66dbed44 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:23.054349 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1817a8-4c1b-4937-a904-936a66dbed44 service glance] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:23.073401 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 731/1475] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:15:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:15:25.127174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15c4b6-ca97-4ccd-a184-c28ae86d6d2b None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:25.127730 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15c4b6-ca97-4ccd-a184-c28ae86d6d2b None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:25.127730 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15c4b6-ca97-4ccd-a184-c28ae86d6d2b None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:25.129089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 745/1476] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:25.134490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c44fee-8b83-4475-b8e0-e4841751ac0e None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:25.134926 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c44fee-8b83-4475-b8e0-e4841751ac0e None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:25.135063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c44fee-8b83-4475-b8e0-e4841751ac0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:25.204996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-18c44fee-8b83-4475-b8e0-e4841751ac0e None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:25.206221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-18c44fee-8b83-4475-b8e0-e4841751ac0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:25.302350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 732/1477] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:15:25.583487 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15c4b6-ca97-4ccd-a184-c28ae86d6d2b None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:25.585050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-ad15c4b6-ca97-4ccd-a184-c28ae86d6d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:25.709531 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15c4b6-ca97-4ccd-a184-c28ae86d6d2b None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:25.758159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:25.803274 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8030637b-6e34-44fb-8c10-47aba64fde20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:25.803876 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030637b-6e34-44fb-8c10-47aba64fde20 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:25.804160 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030637b-6e34-44fb-8c10-47aba64fde20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:25.804429 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030637b-6e34-44fb-8c10-47aba64fde20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:25.805067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8030637b-6e34-44fb-8c10-47aba64fde20 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:25.807755 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: WARNING py.warnings [None req-8030637b-6e34-44fb-8c10-47aba64fde20 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:15:25.807755 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: warnings.warn(msg) Aug 31 00:15:25.807755 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: Aug 31 00:15:25.808916 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8030637b-6e34-44fb-8c10-47aba64fde20 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:25.829253 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 746/1478] 10.222.0.67 () {56 vars in 1108 bytes} [Wed Aug 31 00:15:25 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:15:27.823027 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2321901-89da-488a-8e46-1ad67e897fbe None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:27.823677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2321901-89da-488a-8e46-1ad67e897fbe None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:27.823879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2321901-89da-488a-8e46-1ad67e897fbe None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:27.825319 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 733/1479] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:15:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:27.830796 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376eda51-ddd3-4dd3-aed9-96946c46346d None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:27.831289 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376eda51-ddd3-4dd3-aed9-96946c46346d None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:27.831404 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376eda51-ddd3-4dd3-aed9-96946c46346d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:27.907979 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-376eda51-ddd3-4dd3-aed9-96946c46346d None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:27.909210 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-376eda51-ddd3-4dd3-aed9-96946c46346d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:28.010578 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 747/1480] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:15:28.324013 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2321901-89da-488a-8e46-1ad67e897fbe None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:28.325653 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-e2321901-89da-488a-8e46-1ad67e897fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:28.335303 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:28.474705 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2321901-89da-488a-8e46-1ad67e897fbe None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:28.564694 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:28.597338 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c3235-4753-49e9-b2bd-ad6505cf0189 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:28.597952 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c3235-4753-49e9-b2bd-ad6505cf0189 service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:28.598228 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c3235-4753-49e9-b2bd-ad6505cf0189 service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:28.598494 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c3235-4753-49e9-b2bd-ad6505cf0189 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:28.600129 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-a66c3235-4753-49e9-b2bd-ad6505cf0189 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:28.689221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c3235-4753-49e9-b2bd-ad6505cf0189 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:28.690801 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c3235-4753-49e9-b2bd-ad6505cf0189 service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:28.708943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 734/1481] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:15:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:15:30.801859 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01360455-3302-4dc6-80c4-61be3dc9d94a None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:30.802396 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01360455-3302-4dc6-80c4-61be3dc9d94a None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:30.802396 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01360455-3302-4dc6-80c4-61be3dc9d94a None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:30.803765 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 748/1482] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:30.809623 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e1ca4-6797-4527-8aa4-d80672e3a655 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:30.810136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e1ca4-6797-4527-8aa4-d80672e3a655 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:30.810245 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e1ca4-6797-4527-8aa4-d80672e3a655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:30.875701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-757e1ca4-6797-4527-8aa4-d80672e3a655 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:30.876895 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-757e1ca4-6797-4527-8aa4-d80672e3a655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:30.923286 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:30.973658 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 735/1483] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:15:31.261840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01360455-3302-4dc6-80c4-61be3dc9d94a None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:31.264308 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-01360455-3302-4dc6-80c4-61be3dc9d94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:31.286912 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:31.368499 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01360455-3302-4dc6-80c4-61be3dc9d94a None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:31.441652 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c63a44-b1d6-4c9e-aa34-ac09ac2f1eec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:31.442174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c63a44-b1d6-4c9e-aa34-ac09ac2f1eec service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:31.442450 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c63a44-b1d6-4c9e-aa34-ac09ac2f1eec service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:31.442716 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c63a44-b1d6-4c9e-aa34-ac09ac2f1eec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:31.444345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-69c63a44-b1d6-4c9e-aa34-ac09ac2f1eec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:31.519598 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:31.554454 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c63a44-b1d6-4c9e-aa34-ac09ac2f1eec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:31.555954 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c63a44-b1d6-4c9e-aa34-ac09ac2f1eec service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:31.575315 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 749/1484] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:15:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:15:33.531626 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc529ec5-558e-43d8-ac9e-57c2540cce07 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:33.532197 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc529ec5-558e-43d8-ac9e-57c2540cce07 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:33.532197 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc529ec5-558e-43d8-ac9e-57c2540cce07 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:33.533546 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 736/1485] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:33.538739 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d5044-7767-46cc-9d37-581e2e40d02d None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:33.539217 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d5044-7767-46cc-9d37-581e2e40d02d None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:33.539326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d5044-7767-46cc-9d37-581e2e40d02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:33.607911 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-4b5d5044-7767-46cc-9d37-581e2e40d02d None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:33.609125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4b5d5044-7767-46cc-9d37-581e2e40d02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:33.706427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 750/1486] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:15:33.998378 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc529ec5-558e-43d8-ac9e-57c2540cce07 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:33.999943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-cc529ec5-558e-43d8-ac9e-57c2540cce07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:34.103453 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc529ec5-558e-43d8-ac9e-57c2540cce07 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:34.174514 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:34.176548 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10870aa-23c6-447f-83cb-55c07a5deb57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:34.177242 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10870aa-23c6-447f-83cb-55c07a5deb57 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:34.177749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10870aa-23c6-447f-83cb-55c07a5deb57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:34.178318 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10870aa-23c6-447f-83cb-55c07a5deb57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:34.180187 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b10870aa-23c6-447f-83cb-55c07a5deb57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:34.291650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10870aa-23c6-447f-83cb-55c07a5deb57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:34.293005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10870aa-23c6-447f-83cb-55c07a5deb57 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:34.311441 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 737/1487] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:15:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:15:36.382626 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc336393-a1f5-45f8-9b19-5a4ff473ce21 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:36.383216 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc336393-a1f5-45f8-9b19-5a4ff473ce21 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:36.383216 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc336393-a1f5-45f8-9b19-5a4ff473ce21 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:36.384700 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 751/1488] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:36.390184 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feed6c7-9da3-43f7-969f-9e08fb5948b4 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:36.390624 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feed6c7-9da3-43f7-969f-9e08fb5948b4 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:36.390739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feed6c7-9da3-43f7-969f-9e08fb5948b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:36.456543 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-4feed6c7-9da3-43f7-969f-9e08fb5948b4 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:36.457754 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-4feed6c7-9da3-43f7-969f-9e08fb5948b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:36.512440 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:36.558342 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 738/1489] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:15:36.848524 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc336393-a1f5-45f8-9b19-5a4ff473ce21 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:36.850127 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-fc336393-a1f5-45f8-9b19-5a4ff473ce21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:36.887150 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:36.963599 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc336393-a1f5-45f8-9b19-5a4ff473ce21 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:37.039550 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb8c2b-24b9-4e02-ad02-c470c3f35151 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:37.040401 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb8c2b-24b9-4e02-ad02-c470c3f35151 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:37.040789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb8c2b-24b9-4e02-ad02-c470c3f35151 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:37.041069 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb8c2b-24b9-4e02-ad02-c470c3f35151 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:37.043006 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4fdb8c2b-24b9-4e02-ad02-c470c3f35151 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:37.124319 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:37.153583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb8c2b-24b9-4e02-ad02-c470c3f35151 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:37.155016 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb8c2b-24b9-4e02-ad02-c470c3f35151 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:37.172617 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 752/1490] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:15:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:15:39.316624 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a636e-3d8f-44ec-8061-5393dca7a2c4 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:39.317297 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a636e-3d8f-44ec-8061-5393dca7a2c4 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:39.317594 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a636e-3d8f-44ec-8061-5393dca7a2c4 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:39.318979 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 739/1491] 10.222.0.67 () {54 vars in 927 bytes} [Wed Aug 31 00:15:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:15:39.326336 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb5aae-1ae6-4ca4-be84-2d6d91bbbb54 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:39.326858 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb5aae-1ae6-4ca4-be84-2d6d91bbbb54 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:39.326966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb5aae-1ae6-4ca4-be84-2d6d91bbbb54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:39.395200 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-1fcb5aae-1ae6-4ca4-be84-2d6d91bbbb54 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:15:39.396432 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-1fcb5aae-1ae6-4ca4-be84-2d6d91bbbb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:39.492872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 753/1492] 10.222.0.67 () {58 vars in 1024 bytes} [Wed Aug 31 00:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:15:39.790076 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a636e-3d8f-44ec-8061-5393dca7a2c4 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:15:39.791641 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-562a636e-3d8f-44ec-8061-5393dca7a2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:39.894466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a636e-3d8f-44ec-8061-5393dca7a2c4 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:15:39.967477 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed37b41e-7d1e-4ec5-a523-359a774d9477 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:15:39.968045 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37b41e-7d1e-4ec5-a523-359a774d9477 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:15:39.968319 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37b41e-7d1e-4ec5-a523-359a774d9477 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:15:39.968588 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37b41e-7d1e-4ec5-a523-359a774d9477 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:15:39.970178 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-ed37b41e-7d1e-4ec5-a523-359a774d9477 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:15:39.980184 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:40.089146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed37b41e-7d1e-4ec5-a523-359a774d9477 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:15:40.090567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed37b41e-7d1e-4ec5-a523-359a774d9477 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:15:40.108438 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 740/1493] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:15:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:15:49.491024 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=82 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:15:50.106202 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=50 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:53.275174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285b9b7-80af-44a3-8337-f00e9e39b41f None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:53.275828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285b9b7-80af-44a3-8337-f00e9e39b41f None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:53.275828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285b9b7-80af-44a3-8337-f00e9e39b41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:53.350921 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-8285b9b7-80af-44a3-8337-f00e9e39b41f None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:53.352164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-8285b9b7-80af-44a3-8337-f00e9e39b41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:53.569904 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 754/1494] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:16:53.578005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af4f5f-759a-435b-ba90-8aa73166adf5 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:53.578484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af4f5f-759a-435b-ba90-8aa73166adf5 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:53.578580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af4f5f-759a-435b-ba90-8aa73166adf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:53.658329 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-57af4f5f-759a-435b-ba90-8aa73166adf5 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:53.659564 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-57af4f5f-759a-435b-ba90-8aa73166adf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:53.756017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 741/1495] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:16:53.761041 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285b9b7-80af-44a3-8337-f00e9e39b41f None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:53.762245 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-8285b9b7-80af-44a3-8337-f00e9e39b41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:53.774260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285b9b7-80af-44a3-8337-f00e9e39b41f None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:53.836259 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:53.870434 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcef8aa-44bd-4777-a8fc-2b734567921d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:53.871064 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcef8aa-44bd-4777-a8fc-2b734567921d admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:53.871339 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcef8aa-44bd-4777-a8fc-2b734567921d admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:53.871611 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcef8aa-44bd-4777-a8fc-2b734567921d admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:53.872254 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcef8aa-44bd-4777-a8fc-2b734567921d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:16:53.872555 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcef8aa-44bd-4777-a8fc-2b734567921d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:53.873567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: WARNING py.warnings [None req-0dcef8aa-44bd-4777-a8fc-2b734567921d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:16:53.873567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: warnings.warn(msg) Aug 31 00:16:53.873567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: Aug 31 00:16:53.874452 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcef8aa-44bd-4777-a8fc-2b734567921d admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:53.886208 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 755/1496] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:16:53.894614 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af4f5f-759a-435b-ba90-8aa73166adf5 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:53.904876 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af4f5f-759a-435b-ba90-8aa73166adf5 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:53.966428 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:54.000654 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab893685-0ce6-4d70-8710-cb99c27a6c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:54.001264 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab893685-0ce6-4d70-8710-cb99c27a6c8a admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:54.001543 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab893685-0ce6-4d70-8710-cb99c27a6c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:54.001812 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab893685-0ce6-4d70-8710-cb99c27a6c8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:54.002444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab893685-0ce6-4d70-8710-cb99c27a6c8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:54.004723 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: WARNING py.warnings [None req-ab893685-0ce6-4d70-8710-cb99c27a6c8a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:16:54.004723 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: warnings.warn(msg) Aug 31 00:16:54.004723 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: Aug 31 00:16:54.006308 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab893685-0ce6-4d70-8710-cb99c27a6c8a admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:54.063572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 742/1497] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:16:53 2022] POST /identity/v3/projects => generated 379 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:16:54.068583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcef8aa-44bd-4777-a8fc-2b734567921d admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:54.070102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-0dcef8aa-44bd-4777-a8fc-2b734567921d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:54.104029 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcef8aa-44bd-4777-a8fc-2b734567921d admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:54.200042 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c723c-fe51-4a93-b22d-328933f8d436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:54.200640 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c723c-fe51-4a93-b22d-328933f8d436 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:54.200914 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c723c-fe51-4a93-b22d-328933f8d436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:54.201181 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c723c-fe51-4a93-b22d-328933f8d436 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:54.201809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c723c-fe51-4a93-b22d-328933f8d436 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:54.202806 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: WARNING py.warnings [None req-439c723c-fe51-4a93-b22d-328933f8d436 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:16:54.202806 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: warnings.warn(msg) Aug 31 00:16:54.202806 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: Aug 31 00:16:54.204023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c723c-fe51-4a93-b22d-328933f8d436 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:54.245551 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 756/1498] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:16:54 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:16:54.249696 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab893685-0ce6-4d70-8710-cb99c27a6c8a admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:54.251234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-ab893685-0ce6-4d70-8710-cb99c27a6c8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:54.313360 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab893685-0ce6-4d70-8710-cb99c27a6c8a admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:54.397090 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:54.408389 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ae981-1538-4e75-8d0e-88e9e7e6fbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:54.408991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ae981-1538-4e75-8d0e-88e9e7e6fbf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:54.409266 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ae981-1538-4e75-8d0e-88e9e7e6fbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:54.409528 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ae981-1538-4e75-8d0e-88e9e7e6fbf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:54.410156 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ae981-1538-4e75-8d0e-88e9e7e6fbf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:54.411527 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: WARNING py.warnings [None req-f54ae981-1538-4e75-8d0e-88e9e7e6fbf9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:16:54.411527 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: warnings.warn(msg) Aug 31 00:16:54.411527 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: Aug 31 00:16:54.412397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ae981-1538-4e75-8d0e-88e9e7e6fbf9 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:54.424493 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 743/1499] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:16:54.429233 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c723c-fe51-4a93-b22d-328933f8d436 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:54.430754 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-439c723c-fe51-4a93-b22d-328933f8d436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:54.442597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c723c-fe51-4a93-b22d-328933f8d436 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:54.467574 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:54.538449 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a910fd9-c938-4ff9-9651-8e5405afc06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:54.539130 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a910fd9-c938-4ff9-9651-8e5405afc06e admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:54.539415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a910fd9-c938-4ff9-9651-8e5405afc06e admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:54.539684 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a910fd9-c938-4ff9-9651-8e5405afc06e admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:54.540319 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a910fd9-c938-4ff9-9651-8e5405afc06e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:54.541329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: WARNING py.warnings [None req-3a910fd9-c938-4ff9-9651-8e5405afc06e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:16:54.541329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: warnings.warn(msg) Aug 31 00:16:54.541329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: Aug 31 00:16:54.542211 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a910fd9-c938-4ff9-9651-8e5405afc06e admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:54.555324 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 757/1500] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:16:54.559558 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ae981-1538-4e75-8d0e-88e9e7e6fbf9 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:54.561093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f54ae981-1538-4e75-8d0e-88e9e7e6fbf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:54.595929 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ae981-1538-4e75-8d0e-88e9e7e6fbf9 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:54.692046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4031210-0b73-44ba-a9b7-344f4081c8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:54.692731 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4031210-0b73-44ba-a9b7-344f4081c8ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:54.693009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4031210-0b73-44ba-a9b7-344f4081c8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:54.693280 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4031210-0b73-44ba-a9b7-344f4081c8ec admin admin] PATH_INFO: `/v3/projects/71759a996c8943fb984e5358839df125/users/61bf9c399b8a4c488ffd19da9e122114/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:54.714603 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4031210-0b73-44ba-a9b7-344f4081c8ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=71759a996c8943fb984e5358839df125, user_id=61bf9c399b8a4c488ffd19da9e122114, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:54.715661 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: WARNING py.warnings [None req-f4031210-0b73-44ba-a9b7-344f4081c8ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:16:54.715661 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: warnings.warn(msg) Aug 31 00:16:54.715661 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: Aug 31 00:16:54.717596 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4031210-0b73-44ba-a9b7-344f4081c8ec admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:54.742323 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 744/1501] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:16:54 2022] PUT /identity/v3/projects/71759a996c8943fb984e5358839df125/users/61bf9c399b8a4c488ffd19da9e122114/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:16:54.754856 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a9db4-2b15-45f9-9130-e3f7889ba067 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:54.755202 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a9db4-2b15-45f9-9130-e3f7889ba067 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:54.755487 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a9db4-2b15-45f9-9130-e3f7889ba067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:54.814909 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-f58a9db4-2b15-45f9-9130-e3f7889ba067 None None] MFA Rules not processed for user `61bf9c399b8a4c488ffd19da9e122114`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:54.816127 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f58a9db4-2b15-45f9-9130-e3f7889ba067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:54.891726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 758/1502] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:16:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:16:54.905203 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3160b7-c922-4d80-8d01-eb12224ad610 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:54.905486 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3160b7-c922-4d80-8d01-eb12224ad610 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:54.905760 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3160b7-c922-4d80-8d01-eb12224ad610 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:54.907141 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 745/1503] 10.222.0.67 () {54 vars in 964 bytes} [Wed Aug 31 00:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:54.912320 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ae9fa-2783-4a84-bc3c-68206541093c None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:54.912654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ae9fa-2783-4a84-bc3c-68206541093c None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:54.912925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ae9fa-2783-4a84-bc3c-68206541093c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:54.943924 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:54.988631 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-3f8ae9fa-2783-4a84-bc3c-68206541093c None None] MFA Rules not processed for user `7e42dfb7d22e4435889cb91327db5477`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:54.989848 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-3f8ae9fa-2783-4a84-bc3c-68206541093c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:55.066326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 759/1504] 10.222.0.67 () {58 vars in 1061 bytes} [Wed Aug 31 00:16:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:16:55.075860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3160b7-c922-4d80-8d01-eb12224ad610 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:55.077467 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-dc3160b7-c922-4d80-8d01-eb12224ad610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:55.127818 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:55.184316 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3160b7-c922-4d80-8d01-eb12224ad610 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:55.257089 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12970a84-2b7a-4fdc-9d2d-bb67a23845a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:55.257616 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12970a84-2b7a-4fdc-9d2d-bb67a23845a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:55.257899 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12970a84-2b7a-4fdc-9d2d-bb67a23845a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:55.258174 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12970a84-2b7a-4fdc-9d2d-bb67a23845a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:55.260069 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-12970a84-2b7a-4fdc-9d2d-bb67a23845a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:55.358033 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12970a84-2b7a-4fdc-9d2d-bb67a23845a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:55.359507 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12970a84-2b7a-4fdc-9d2d-bb67a23845a6 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:55.363008 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:55.377939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 746/1505] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:16:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:16:55.424650 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f43e2-5412-4037-9b3a-73ed37f21c44 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:55.424943 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f43e2-5412-4037-9b3a-73ed37f21c44 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:55.425219 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f43e2-5412-4037-9b3a-73ed37f21c44 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:55.426488 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 760/1506] 10.222.0.67 () {52 vars in 830 bytes} [Wed Aug 31 00:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:16:55.431146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12970a84-2b7a-4fdc-9d2d-bb67a23845a6 service cinder] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:55.432321 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-12970a84-2b7a-4fdc-9d2d-bb67a23845a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:55.535534 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12970a84-2b7a-4fdc-9d2d-bb67a23845a6 service cinder] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:55.627768 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248630b-8d9d-40a6-a7eb-8a114adf0d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:55.628465 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248630b-8d9d-40a6-a7eb-8a114adf0d61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:55.628647 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248630b-8d9d-40a6-a7eb-8a114adf0d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:55.628915 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248630b-8d9d-40a6-a7eb-8a114adf0d61 admin admin] PATH_INFO: `/v3/users/61bf9c399b8a4c488ffd19da9e122114` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:55.641612 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e248630b-8d9d-40a6-a7eb-8a114adf0d61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61bf9c399b8a4c488ffd19da9e122114)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:55.642628 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: WARNING py.warnings [None req-e248630b-8d9d-40a6-a7eb-8a114adf0d61 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:16:55.642628 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: warnings.warn(msg) Aug 31 00:16:55.642628 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: Aug 31 00:16:55.643855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e248630b-8d9d-40a6-a7eb-8a114adf0d61 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:55.654023 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:55.713092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e248630b-8d9d-40a6-a7eb-8a114adf0d61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61bf9c399b8a4c488ffd19da9e122114'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:16:55.735704 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e248630b-8d9d-40a6-a7eb-8a114adf0d61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61bf9c399b8a4c488ffd19da9e122114'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:16:55.749338 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e248630b-8d9d-40a6-a7eb-8a114adf0d61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61bf9c399b8a4c488ffd19da9e122114'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:16:55.761884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e248630b-8d9d-40a6-a7eb-8a114adf0d61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61bf9c399b8a4c488ffd19da9e122114'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:16:55.762415 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e248630b-8d9d-40a6-a7eb-8a114adf0d61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61bf9c399b8a4c488ffd19da9e122114'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:16:55.768391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 747/1507] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:16:55 2022] DELETE /identity/v3/users/61bf9c399b8a4c488ffd19da9e122114 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:16:55.773467 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5915be9-03ee-47f1-9ecb-5474ade632d7 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:55.773926 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5915be9-03ee-47f1-9ecb-5474ade632d7 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:55.774019 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5915be9-03ee-47f1-9ecb-5474ade632d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:55.829480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-e5915be9-03ee-47f1-9ecb-5474ade632d7 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:16:55.830668 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e5915be9-03ee-47f1-9ecb-5474ade632d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:55.905899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:55.919169 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 761/1508] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:16:55.929070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248630b-8d9d-40a6-a7eb-8a114adf0d61 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:55.930307 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-e248630b-8d9d-40a6-a7eb-8a114adf0d61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:56.031919 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248630b-8d9d-40a6-a7eb-8a114adf0d61 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:56.087563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:16:56.108575 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277e2cd-7cc0-461b-a36e-0d3fdb01e87b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:56.109185 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277e2cd-7cc0-461b-a36e-0d3fdb01e87b service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:56.109468 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277e2cd-7cc0-461b-a36e-0d3fdb01e87b service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:56.109741 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277e2cd-7cc0-461b-a36e-0d3fdb01e87b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:56.111383 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-4277e2cd-7cc0-461b-a36e-0d3fdb01e87b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:56.223857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4277e2cd-7cc0-461b-a36e-0d3fdb01e87b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:56.226280 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4277e2cd-7cc0-461b-a36e-0d3fdb01e87b service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:56.248049 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 748/1509] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:16:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:16:56.612832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5915be9-03ee-47f1-9ecb-5474ade632d7 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:16:56.614070 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e5915be9-03ee-47f1-9ecb-5474ade632d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:16:56.625761 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5915be9-03ee-47f1-9ecb-5474ade632d7 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:16:56.726156 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d839d11d-c4bd-4021-b6eb-e22de226cc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:16:56.726781 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839d11d-c4bd-4021-b6eb-e22de226cc11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:16:56.727076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839d11d-c4bd-4021-b6eb-e22de226cc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:16:56.727346 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839d11d-c4bd-4021-b6eb-e22de226cc11 admin admin] PATH_INFO: `/v3/projects/71759a996c8943fb984e5358839df125` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:16:56.736989 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d839d11d-c4bd-4021-b6eb-e22de226cc11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71759a996c8943fb984e5358839df125)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:16:56.738009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: WARNING py.warnings [None req-d839d11d-c4bd-4021-b6eb-e22de226cc11 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:16:56.738009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: warnings.warn(msg) Aug 31 00:16:56.738009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: Aug 31 00:16:56.739238 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d839d11d-c4bd-4021-b6eb-e22de226cc11 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:16:56.798402 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d839d11d-c4bd-4021-b6eb-e22de226cc11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71759a996c8943fb984e5358839df125', 'request_id': 'req-d839d11d-c4bd-4021-b6eb-e22de226cc11'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:16:56.804430 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d839d11d-c4bd-4021-b6eb-e22de226cc11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71759a996c8943fb984e5358839df125', 'request_id': 'req-d839d11d-c4bd-4021-b6eb-e22de226cc11'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:16:56.817345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d839d11d-c4bd-4021-b6eb-e22de226cc11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71759a996c8943fb984e5358839df125', 'request_id': 'req-d839d11d-c4bd-4021-b6eb-e22de226cc11'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:16:56.843687 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d839d11d-c4bd-4021-b6eb-e22de226cc11 admin admin] The token cache is being invalidate because project 71759a996c8943fb984e5358839df125 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:16:56.843982 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d839d11d-c4bd-4021-b6eb-e22de226cc11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:16:56.844419 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d839d11d-c4bd-4021-b6eb-e22de226cc11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:16:56.845824 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 762/1510] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:16:56 2022] DELETE /identity/v3/projects/71759a996c8943fb984e5358839df125 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:17:06.245755 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=62 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:17:06.841542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:18:27.697344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a6bd3-090b-4bac-8c8e-4b6ca5b15551 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:27.697955 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a6bd3-090b-4bac-8c8e-4b6ca5b15551 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:27.697955 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a6bd3-090b-4bac-8c8e-4b6ca5b15551 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:27.699301 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 749/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 00:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:18:27.704311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679606d-b641-48b0-bffa-cb251e1a0f3f None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:27.704776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679606d-b641-48b0-bffa-cb251e1a0f3f None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:27.704889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679606d-b641-48b0-bffa-cb251e1a0f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:27.785811 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-7679606d-b641-48b0-bffa-cb251e1a0f3f None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:18:27.787057 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-7679606d-b641-48b0-bffa-cb251e1a0f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:18:27.893273 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 763/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 00:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:18:28.052974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a6bd3-090b-4bac-8c8e-4b6ca5b15551 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:18:28.055536 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-a18a6bd3-090b-4bac-8c8e-4b6ca5b15551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:18:28.173715 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a6bd3-090b-4bac-8c8e-4b6ca5b15551 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:18:28.179347 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91815f3-f955-4da8-b2f6-8bc3a1f15dd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:18:28.179954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91815f3-f955-4da8-b2f6-8bc3a1f15dd9 service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:28.180234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91815f3-f955-4da8-b2f6-8bc3a1f15dd9 service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:28.180507 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91815f3-f955-4da8-b2f6-8bc3a1f15dd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:28.188652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91815f3-f955-4da8-b2f6-8bc3a1f15dd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:18:28.190229 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91815f3-f955-4da8-b2f6-8bc3a1f15dd9 service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:18:28.199750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 750/1513] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:18:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:18:29.034301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d1781-f872-49f3-926d-a18f68d3b284 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:29.034803 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d1781-f872-49f3-926d-a18f68d3b284 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:29.034918 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d1781-f872-49f3-926d-a18f68d3b284 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:29.036237 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 764/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 00:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:18:29.041187 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a59673-44a3-4838-939c-b000a42c236b None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:29.041618 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a59673-44a3-4838-939c-b000a42c236b None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:29.041719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a59673-44a3-4838-939c-b000a42c236b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:29.060856 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-50a59673-44a3-4838-939c-b000a42c236b None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:18:29.062045 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-50a59673-44a3-4838-939c-b000a42c236b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:18:29.070413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 751/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 00:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:18:29.224057 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d1781-f872-49f3-926d-a18f68d3b284 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:18:29.238143 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d1781-f872-49f3-926d-a18f68d3b284 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:18:29.243548 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646454dc-eec8-4b8c-9f30-089046fc26a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:18:29.244142 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646454dc-eec8-4b8c-9f30-089046fc26a1 service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:29.244420 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646454dc-eec8-4b8c-9f30-089046fc26a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:29.244693 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646454dc-eec8-4b8c-9f30-089046fc26a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:29.253467 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646454dc-eec8-4b8c-9f30-089046fc26a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:18:29.255059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646454dc-eec8-4b8c-9f30-089046fc26a1 service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:18:29.264494 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 765/1516] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:18:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:18:29.807828 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d660dd2-17d7-4c1a-ba27-6db0df8e9450 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:29.808116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d660dd2-17d7-4c1a-ba27-6db0df8e9450 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:29.808409 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d660dd2-17d7-4c1a-ba27-6db0df8e9450 None None] PATH_INFO: `/v3` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:29.809735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 752/1517] 10.222.0.58 () {54 vars in 925 bytes} [Wed Aug 31 00:18:29 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:18:29.816017 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7796f30-f412-403c-8ba0-743e522dfb81 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:29.816240 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7796f30-f412-403c-8ba0-743e522dfb81 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:29.816512 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7796f30-f412-403c-8ba0-743e522dfb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:29.878406 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-e7796f30-f412-403c-8ba0-743e522dfb81 None None] MFA Rules not processed for user `fdacdcb2451742938164fdd44cdbaebe`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:18:29.879650 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e7796f30-f412-403c-8ba0-743e522dfb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:18:29.896495 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:18:29.955112 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 766/1518] 10.222.0.58 () {58 vars in 1015 bytes} [Wed Aug 31 00:18:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:18:39.055912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=56 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:18:39.070264 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228094c6-a8b2-4453-a9c6-1c7c6507bbd0 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:39.070547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228094c6-a8b2-4453-a9c6-1c7c6507bbd0 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:39.070820 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228094c6-a8b2-4453-a9c6-1c7c6507bbd0 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:39.072191 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 753/1519] 10.222.0.67 () {54 vars in 961 bytes} [Wed Aug 31 00:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:18:39.077193 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07300124-5a08-4808-8e8f-fa096b4c9cb0 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:39.077674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07300124-5a08-4808-8e8f-fa096b4c9cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:39.077785 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07300124-5a08-4808-8e8f-fa096b4c9cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:39.097468 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-07300124-5a08-4808-8e8f-fa096b4c9cb0 None None] MFA Rules not processed for user `fdacdcb2451742938164fdd44cdbaebe`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:18:39.098667 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-07300124-5a08-4808-8e8f-fa096b4c9cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:18:39.107955 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 767/1520] 10.222.0.67 () {58 vars in 1058 bytes} [Wed Aug 31 00:18:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:18:39.115540 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228094c6-a8b2-4453-a9c6-1c7c6507bbd0 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:18:39.128906 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228094c6-a8b2-4453-a9c6-1c7c6507bbd0 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:18:39.134608 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb56a2-9bdc-49bf-bfd3-bfbe13d48258 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdacdcb2451742938164fdd44cdbaebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:18:39.135235 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb56a2-9bdc-49bf-bfd3-bfbe13d48258 service placement] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:39.135532 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb56a2-9bdc-49bf-bfd3-bfbe13d48258 service placement] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:39.135796 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb56a2-9bdc-49bf-bfd3-bfbe13d48258 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:39.144197 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb56a2-9bdc-49bf-bfd3-bfbe13d48258 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:18:39.145785 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb56a2-9bdc-49bf-bfd3-bfbe13d48258 service placement] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:18:39.156300 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 754/1521] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:18:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:18:40.502031 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9d55a-e3d7-462e-9325-14f655798410 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:40.502613 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9d55a-e3d7-462e-9325-14f655798410 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:40.502613 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9d55a-e3d7-462e-9325-14f655798410 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:40.504022 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 768/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 00:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:18:40.510046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cccf7-d0c1-4c1e-bfed-505796778c72 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:40.510458 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cccf7-d0c1-4c1e-bfed-505796778c72 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:40.510577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cccf7-d0c1-4c1e-bfed-505796778c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:40.529441 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-693cccf7-d0c1-4c1e-bfed-505796778c72 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:18:40.530613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-693cccf7-d0c1-4c1e-bfed-505796778c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:18:40.538354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 755/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 00:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:18:40.694090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9d55a-e3d7-462e-9325-14f655798410 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:18:40.708623 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9d55a-e3d7-462e-9325-14f655798410 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:18:40.714173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c7c45c-a302-4e0f-8744-02e047a54773 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:18:40.714759 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7c45c-a302-4e0f-8744-02e047a54773 service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:40.715054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7c45c-a302-4e0f-8744-02e047a54773 service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:40.715335 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7c45c-a302-4e0f-8744-02e047a54773 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:40.723351 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c7c45c-a302-4e0f-8744-02e047a54773 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:18:40.724976 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c7c45c-a302-4e0f-8744-02e047a54773 service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:18:40.734261 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 769/1524] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:18:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:18:41.563694 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5effb-a505-44a6-b470-986aea52072f None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:41.564241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5effb-a505-44a6-b470-986aea52072f None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:41.564341 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5effb-a505-44a6-b470-986aea52072f None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:41.565623 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 756/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 00:18:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:18:41.571115 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2fff9-e667-403c-8583-281423e9a453 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:41.571529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2fff9-e667-403c-8583-281423e9a453 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:41.571661 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2fff9-e667-403c-8583-281423e9a453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:41.591646 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-66f2fff9-e667-403c-8583-281423e9a453 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:18:41.592826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-66f2fff9-e667-403c-8583-281423e9a453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:18:41.601013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 770/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 00:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:18:41.752946 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d5effb-a505-44a6-b470-986aea52072f None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:18:41.768389 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d5effb-a505-44a6-b470-986aea52072f None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:18:41.774796 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bacc8d-267f-4b9a-aa73-178b2c223305 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:18:41.775452 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bacc8d-267f-4b9a-aa73-178b2c223305 service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:41.775730 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bacc8d-267f-4b9a-aa73-178b2c223305 service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:41.776002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bacc8d-267f-4b9a-aa73-178b2c223305 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:41.784394 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bacc8d-267f-4b9a-aa73-178b2c223305 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:18:41.785968 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bacc8d-267f-4b9a-aa73-178b2c223305 service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:18:41.795562 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 757/1527] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:18:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:18:46.249917 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0df666-c7d7-4458-95c3-62d2b5607f28 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:46.250455 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0df666-c7d7-4458-95c3-62d2b5607f28 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:46.250560 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0df666-c7d7-4458-95c3-62d2b5607f28 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:46.251916 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 771/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:18:46.258005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264d51d-179d-4c06-b7e2-ec893e2ba914 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:46.258420 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264d51d-179d-4c06-b7e2-ec893e2ba914 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:46.258547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264d51d-179d-4c06-b7e2-ec893e2ba914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:46.278148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-2264d51d-179d-4c06-b7e2-ec893e2ba914 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:18:46.279367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-2264d51d-179d-4c06-b7e2-ec893e2ba914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:18:46.287858 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 758/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:18:46.572697 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0df666-c7d7-4458-95c3-62d2b5607f28 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:18:46.574429 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-ed0df666-c7d7-4458-95c3-62d2b5607f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:18:46.652853 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0df666-c7d7-4458-95c3-62d2b5607f28 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:18:46.656467 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9fbb62-bc65-464f-b269-098901467688 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:18:46.657073 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fbb62-bc65-464f-b269-098901467688 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:46.657352 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fbb62-bc65-464f-b269-098901467688 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:46.657622 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fbb62-bc65-464f-b269-098901467688 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:46.667473 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9fbb62-bc65-464f-b269-098901467688 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:18:46.668905 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9fbb62-bc65-464f-b269-098901467688 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:18:46.678289 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 772/1530] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:18:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:18:48.648819 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71d03e-f9ba-48f4-8a95-1aa828a2a853 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:48.649346 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71d03e-f9ba-48f4-8a95-1aa828a2a853 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:48.649445 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71d03e-f9ba-48f4-8a95-1aa828a2a853 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:48.650830 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 759/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:18:48.656199 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02740740-3227-42f9-af81-621022f690ae None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:48.656600 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02740740-3227-42f9-af81-621022f690ae None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:48.656792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02740740-3227-42f9-af81-621022f690ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:48.676080 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-02740740-3227-42f9-af81-621022f690ae None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:18:48.677878 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-02740740-3227-42f9-af81-621022f690ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:18:48.685807 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 773/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:18:48.980392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71d03e-f9ba-48f4-8a95-1aa828a2a853 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:18:48.992000 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71d03e-f9ba-48f4-8a95-1aa828a2a853 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:18:48.995577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b4802-a445-490e-8ba8-a55a03a2713b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:18:48.996177 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b4802-a445-490e-8ba8-a55a03a2713b service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:48.996455 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b4802-a445-490e-8ba8-a55a03a2713b service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:48.996724 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b4802-a445-490e-8ba8-a55a03a2713b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:49.006675 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b4802-a445-490e-8ba8-a55a03a2713b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:18:49.008388 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b4802-a445-490e-8ba8-a55a03a2713b service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:18:49.019049 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 760/1533] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:18:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:18:52.598439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868f44d-d8bc-481e-82e3-baf3231ed32d None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:52.599109 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868f44d-d8bc-481e-82e3-baf3231ed32d None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:52.599109 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868f44d-d8bc-481e-82e3-baf3231ed32d None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:52.600389 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 774/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:18:52.605702 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5b6be-cb7a-4d42-a7f5-8ce80a67d3f7 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:52.606135 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5b6be-cb7a-4d42-a7f5-8ce80a67d3f7 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:52.606284 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5b6be-cb7a-4d42-a7f5-8ce80a67d3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:52.626967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-55d5b6be-cb7a-4d42-a7f5-8ce80a67d3f7 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:18:52.628190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-55d5b6be-cb7a-4d42-a7f5-8ce80a67d3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:18:52.637645 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 761/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:18:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:18:53.197827 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868f44d-d8bc-481e-82e3-baf3231ed32d None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:18:53.199520 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-8868f44d-d8bc-481e-82e3-baf3231ed32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:18:53.275432 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:18:53.289387 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868f44d-d8bc-481e-82e3-baf3231ed32d None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:18:53.292939 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37f5ad-3686-4a21-9a93-28cba0b96f56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:18:53.293545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37f5ad-3686-4a21-9a93-28cba0b96f56 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:53.293823 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37f5ad-3686-4a21-9a93-28cba0b96f56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:53.294095 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37f5ad-3686-4a21-9a93-28cba0b96f56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:53.303462 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d37f5ad-3686-4a21-9a93-28cba0b96f56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:18:53.304884 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d37f5ad-3686-4a21-9a93-28cba0b96f56 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:18:53.314106 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 775/1536] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:18:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:18:55.392829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32eab05-69e6-467b-b965-9cffce9e2dff None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:55.393369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32eab05-69e6-467b-b965-9cffce9e2dff None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:55.393369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32eab05-69e6-467b-b965-9cffce9e2dff None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:55.394717 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 762/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 00:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:18:55.400368 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c6e3c-8217-4a1c-85b8-89144557780f None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:55.400797 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c6e3c-8217-4a1c-85b8-89144557780f None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:55.400930 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c6e3c-8217-4a1c-85b8-89144557780f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:55.421264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-e51c6e3c-8217-4a1c-85b8-89144557780f None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:18:55.422475 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e51c6e3c-8217-4a1c-85b8-89144557780f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:18:55.430469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 776/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 00:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:18:55.980027 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32eab05-69e6-467b-b965-9cffce9e2dff None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:18:55.981676 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-e32eab05-69e6-467b-b965-9cffce9e2dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:18:55.996309 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32eab05-69e6-467b-b965-9cffce9e2dff None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:18:56.000077 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e643c61-e5bc-42e4-8095-1815b9e4fbae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:18:56.000688 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e643c61-e5bc-42e4-8095-1815b9e4fbae service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:18:56.000966 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e643c61-e5bc-42e4-8095-1815b9e4fbae service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:18:56.001233 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e643c61-e5bc-42e4-8095-1815b9e4fbae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:18:56.010763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e643c61-e5bc-42e4-8095-1815b9e4fbae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:18:56.012225 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e643c61-e5bc-42e4-8095-1815b9e4fbae service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:18:56.021983 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 763/1539] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:18:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:19:05.416250 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=11 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:19:06.017472 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=33 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:24.311361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38986ff-5d94-4a76-b8ee-804bcf5edb35 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:24.312383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38986ff-5d94-4a76-b8ee-804bcf5edb35 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:24.312383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38986ff-5d94-4a76-b8ee-804bcf5edb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:24.344052 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-b38986ff-5d94-4a76-b8ee-804bcf5edb35 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:24.345495 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b38986ff-5d94-4a76-b8ee-804bcf5edb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:24.366362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 777/1540] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:24.375915 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063d9d0-3fb9-4225-9f37-2f82d8670ad6 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:24.376816 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063d9d0-3fb9-4225-9f37-2f82d8670ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:24.376816 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063d9d0-3fb9-4225-9f37-2f82d8670ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:24.413289 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-9063d9d0-3fb9-4225-9f37-2f82d8670ad6 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:24.414527 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-9063d9d0-3fb9-4225-9f37-2f82d8670ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:24.428130 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 764/1541] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:24.430567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320f45e-3ec8-4778-a36a-9b5c03b4f28c None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:24.430862 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320f45e-3ec8-4778-a36a-9b5c03b4f28c None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:24.431172 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320f45e-3ec8-4778-a36a-9b5c03b4f28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:24.433006 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9063d9d0-3fb9-4225-9f37-2f82d8670ad6 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:24.443113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9063d9d0-3fb9-4225-9f37-2f82d8670ad6 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:24.460423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86357cd7-30e0-4dbb-b7bd-07b273fc3c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:24.461058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86357cd7-30e0-4dbb-b7bd-07b273fc3c13 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:24.461587 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86357cd7-30e0-4dbb-b7bd-07b273fc3c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:24.461913 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86357cd7-30e0-4dbb-b7bd-07b273fc3c13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:24.462519 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86357cd7-30e0-4dbb-b7bd-07b273fc3c13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:20:24.462971 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86357cd7-30e0-4dbb-b7bd-07b273fc3c13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:24.464050 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: WARNING py.warnings [None req-86357cd7-30e0-4dbb-b7bd-07b273fc3c13 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:20:24.464050 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: warnings.warn(msg) Aug 31 00:20:24.464050 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: Aug 31 00:20:24.464948 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86357cd7-30e0-4dbb-b7bd-07b273fc3c13 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:24.465918 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-1320f45e-3ec8-4778-a36a-9b5c03b4f28c None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:24.467357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-1320f45e-3ec8-4778-a36a-9b5c03b4f28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:24.482538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 765/1542] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:20:24.490969 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86357cd7-30e0-4dbb-b7bd-07b273fc3c13 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:24.500830 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 778/1543] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:24.505494 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86357cd7-30e0-4dbb-b7bd-07b273fc3c13 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:24.507719 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31a202-4626-415c-9ce0-09ae81748b4d None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:24.508280 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31a202-4626-415c-9ce0-09ae81748b4d None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:24.508484 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31a202-4626-415c-9ce0-09ae81748b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:24.514447 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130d731-59b3-4477-bc51-6a1a70d124a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:24.515654 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130d731-59b3-4477-bc51-6a1a70d124a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:24.515941 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130d731-59b3-4477-bc51-6a1a70d124a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:24.516279 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130d731-59b3-4477-bc51-6a1a70d124a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:24.516921 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130d731-59b3-4477-bc51-6a1a70d124a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:24.519872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130d731-59b3-4477-bc51-6a1a70d124a3 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:24.557769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-1c31a202-4626-415c-9ce0-09ae81748b4d None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:24.560105 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-1c31a202-4626-415c-9ce0-09ae81748b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:24.580454 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 779/1544] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:20:24.581549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd5900-0526-4fd8-9eaa-b9aac36d2f3a None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:24.581836 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd5900-0526-4fd8-9eaa-b9aac36d2f3a None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:24.582105 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd5900-0526-4fd8-9eaa-b9aac36d2f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:24.607546 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 766/1545] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:20:24 2022] POST /identity/v3/projects => generated 403 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:20:24.609245 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a4ade-f4da-43fa-8588-802656f5d41c None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:24.609536 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a4ade-f4da-43fa-8588-802656f5d41c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:24.609813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a4ade-f4da-43fa-8588-802656f5d41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:24.618932 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-30cd5900-0526-4fd8-9eaa-b9aac36d2f3a None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:24.620632 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-30cd5900-0526-4fd8-9eaa-b9aac36d2f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:24.632857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-e71a4ade-f4da-43fa-8588-802656f5d41c None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:24.634080 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-e71a4ade-f4da-43fa-8588-802656f5d41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:24.697845 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 780/1546] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:24.699147 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791839c3-5d03-4f65-8b60-b374831e83d0 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:24.699440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791839c3-5d03-4f65-8b60-b374831e83d0 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:24.699791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791839c3-5d03-4f65-8b60-b374831e83d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:24.718282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 767/1547] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:24.719555 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba782f-dbdf-4bd9-b17d-16ffd59b6f54 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:24.719845 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba782f-dbdf-4bd9-b17d-16ffd59b6f54 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:24.720126 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba782f-dbdf-4bd9-b17d-16ffd59b6f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:24.720249 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-791839c3-5d03-4f65-8b60-b374831e83d0 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:24.721392 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-791839c3-5d03-4f65-8b60-b374831e83d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:24.729222 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 781/1548] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:24.730475 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b2464-1016-4e9f-862e-412946226f94 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:24.730684 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b2464-1016-4e9f-862e-412946226f94 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:24.730953 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b2464-1016-4e9f-862e-412946226f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:24.741747 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-75ba782f-dbdf-4bd9-b17d-16ffd59b6f54 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:24.743371 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-75ba782f-dbdf-4bd9-b17d-16ffd59b6f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:24.751010 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-819b2464-1016-4e9f-862e-412946226f94 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:24.751443 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 768/1549] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:24.752373 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-819b2464-1016-4e9f-862e-412946226f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:24.755885 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba782f-dbdf-4bd9-b17d-16ffd59b6f54 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:24.761475 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 782/1550] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:24.763500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d87a44-e9c9-42bb-99c9-a25a265a6f70 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:24.763789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d87a44-e9c9-42bb-99c9-a25a265a6f70 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:24.764063 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d87a44-e9c9-42bb-99c9-a25a265a6f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:24.767620 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba782f-dbdf-4bd9-b17d-16ffd59b6f54 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:24.772343 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08693c3-c822-41a8-80fe-51a77b4780e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:24.773017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08693c3-c822-41a8-80fe-51a77b4780e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:24.773312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08693c3-c822-41a8-80fe-51a77b4780e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:24.773571 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08693c3-c822-41a8-80fe-51a77b4780e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:24.774326 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08693c3-c822-41a8-80fe-51a77b4780e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:24.775368 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: WARNING py.warnings [None req-a08693c3-c822-41a8-80fe-51a77b4780e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:20:24.775368 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: warnings.warn(msg) Aug 31 00:20:24.775368 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: Aug 31 00:20:24.776565 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08693c3-c822-41a8-80fe-51a77b4780e4 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:24.783450 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-93d87a44-e9c9-42bb-99c9-a25a265a6f70 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:24.784639 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-93d87a44-e9c9-42bb-99c9-a25a265a6f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:24.793839 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 783/1551] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:24.794833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ec4c2-0f48-4aee-8571-faffb28ee7ad None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:24.795079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ec4c2-0f48-4aee-8571-faffb28ee7ad None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:24.795772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ec4c2-0f48-4aee-8571-faffb28ee7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:24.806579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 769/1552] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:20:24 2022] POST /identity/v3/users => generated 302 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:24.807932 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08693c3-c822-41a8-80fe-51a77b4780e4 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:24.816293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-a62ec4c2-0f48-4aee-8571-faffb28ee7ad None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:24.817478 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a62ec4c2-0f48-4aee-8571-faffb28ee7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:24.818157 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08693c3-c822-41a8-80fe-51a77b4780e4 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:24.824041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed652b22-2044-4a5a-aa48-a240db1c8606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:24.824647 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed652b22-2044-4a5a-aa48-a240db1c8606 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:24.824929 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed652b22-2044-4a5a-aa48-a240db1c8606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:24.825202 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed652b22-2044-4a5a-aa48-a240db1c8606 admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:24.825847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed652b22-2044-4a5a-aa48-a240db1c8606 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:20:24.826124 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 784/1553] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:24.830739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed652b22-2044-4a5a-aa48-a240db1c8606 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:24.830851 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a58d0e-c433-4e2b-8b93-6b4c9d2af13f None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:24.830851 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a58d0e-c433-4e2b-8b93-6b4c9d2af13f None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:24.830851 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a58d0e-c433-4e2b-8b93-6b4c9d2af13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:24.831106 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed652b22-2044-4a5a-aa48-a240db1c8606 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:24.842217 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 770/1554] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:20:24.842217 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce125683-3d36-47df-95be-bd676f25ce0b None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:24.842217 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce125683-3d36-47df-95be-bd676f25ce0b None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:24.842217 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce125683-3d36-47df-95be-bd676f25ce0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:24.849676 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-22a58d0e-c433-4e2b-8b93-6b4c9d2af13f None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:24.849940 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-22a58d0e-c433-4e2b-8b93-6b4c9d2af13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:24.858653 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 785/1555] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:24.860090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f86370-7da7-486c-8423-744be4e54742 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:24.860501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f86370-7da7-486c-8423-744be4e54742 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:24.861003 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f86370-7da7-486c-8423-744be4e54742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:24.863605 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-ce125683-3d36-47df-95be-bd676f25ce0b None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:24.865209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-ce125683-3d36-47df-95be-bd676f25ce0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:24.873138 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 771/1556] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:24.873921 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce125683-3d36-47df-95be-bd676f25ce0b None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:24.883025 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-12f86370-7da7-486c-8423-744be4e54742 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:24.884586 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-12f86370-7da7-486c-8423-744be4e54742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:24.886799 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce125683-3d36-47df-95be-bd676f25ce0b None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:24.891298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa664699-d84e-4fd2-a2e1-4ebcb3a2ab27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:24.891890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa664699-d84e-4fd2-a2e1-4ebcb3a2ab27 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:24.892170 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa664699-d84e-4fd2-a2e1-4ebcb3a2ab27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:24.892447 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa664699-d84e-4fd2-a2e1-4ebcb3a2ab27 admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:24.892552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 786/1557] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:24.893148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa664699-d84e-4fd2-a2e1-4ebcb3a2ab27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:20:24.893567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa664699-d84e-4fd2-a2e1-4ebcb3a2ab27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:24.893672 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d50228-a55a-48d8-8425-8ccbdb058ccc None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:24.893962 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d50228-a55a-48d8-8425-8ccbdb058ccc None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:24.894241 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d50228-a55a-48d8-8425-8ccbdb058ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:24.895187 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa664699-d84e-4fd2-a2e1-4ebcb3a2ab27 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:24.907163 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 772/1558] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:20:24.908718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa664699-d84e-4fd2-a2e1-4ebcb3a2ab27 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:24.914477 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-c2d50228-a55a-48d8-8425-8ccbdb058ccc None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:24.915773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-c2d50228-a55a-48d8-8425-8ccbdb058ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:24.919507 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa664699-d84e-4fd2-a2e1-4ebcb3a2ab27 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:24.923743 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 787/1559] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:24.924127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abea4d-e1da-47a6-adc9-402553d1b3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:24.924717 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abea4d-e1da-47a6-adc9-402553d1b3b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:24.924912 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6642788-82c0-4612-bb5c-6e71cd08eb54 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:24.925012 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abea4d-e1da-47a6-adc9-402553d1b3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:24.925198 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6642788-82c0-4612-bb5c-6e71cd08eb54 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:24.925294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abea4d-e1da-47a6-adc9-402553d1b3b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:24.925472 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6642788-82c0-4612-bb5c-6e71cd08eb54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:24.925925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7abea4d-e1da-47a6-adc9-402553d1b3b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:24.927707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7abea4d-e1da-47a6-adc9-402553d1b3b7 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:24.938728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 773/1560] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:24.939538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abea4d-e1da-47a6-adc9-402553d1b3b7 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:24.948337 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-f6642788-82c0-4612-bb5c-6e71cd08eb54 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:24.949528 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f6642788-82c0-4612-bb5c-6e71cd08eb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:24.949810 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abea4d-e1da-47a6-adc9-402553d1b3b7 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:24.954258 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d14509-0058-4c5e-bec0-c8a2234f34cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:24.954838 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d14509-0058-4c5e-bec0-c8a2234f34cf admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:24.955140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d14509-0058-4c5e-bec0-c8a2234f34cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:24.955411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d14509-0058-4c5e-bec0-c8a2234f34cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:24.956050 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d14509-0058-4c5e-bec0-c8a2234f34cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:24.958018 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 788/1561] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:24.958140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d14509-0058-4c5e-bec0-c8a2234f34cf admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:24.958940 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6642788-82c0-4612-bb5c-6e71cd08eb54 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:24.968644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6642788-82c0-4612-bb5c-6e71cd08eb54 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:24.973343 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e7180-4956-4781-868d-75b23fa9f368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:24.973945 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e7180-4956-4781-868d-75b23fa9f368 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:24.974256 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e7180-4956-4781-868d-75b23fa9f368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:24.974493 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e7180-4956-4781-868d-75b23fa9f368 admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:24.975937 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e7180-4956-4781-868d-75b23fa9f368 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:20:24.976294 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e7180-4956-4781-868d-75b23fa9f368 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:24.978258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e7180-4956-4781-868d-75b23fa9f368 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:24.987970 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 774/1562] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:20:24 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:24.989041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d14509-0058-4c5e-bec0-c8a2234f34cf admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:24.994518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 789/1563] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:20:24.995868 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e7180-4956-4781-868d-75b23fa9f368 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.006079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e7180-4956-4781-868d-75b23fa9f368 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.017548 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:25.059939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d14509-0058-4c5e-bec0-c8a2234f34cf admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.064232 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90d8a1-917f-4730-a0f8-445a186ff92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.065002 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90d8a1-917f-4730-a0f8-445a186ff92d admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.065310 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90d8a1-917f-4730-a0f8-445a186ff92d admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.065731 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90d8a1-917f-4730-a0f8-445a186ff92d admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.066406 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d5d4fa-4de9-4fb9-91c3-9dbfdaf98757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.066588 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed90d8a1-917f-4730-a0f8-445a186ff92d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:20:25.066997 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed90d8a1-917f-4730-a0f8-445a186ff92d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.067153 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5d4fa-4de9-4fb9-91c3-9dbfdaf98757 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.074528 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5d4fa-4de9-4fb9-91c3-9dbfdaf98757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.074528 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5d4fa-4de9-4fb9-91c3-9dbfdaf98757 admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.074528 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d5d4fa-4de9-4fb9-91c3-9dbfdaf98757 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:20:25.074528 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d5d4fa-4de9-4fb9-91c3-9dbfdaf98757 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.074956 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed90d8a1-917f-4730-a0f8-445a186ff92d admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:25.075096 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d5d4fa-4de9-4fb9-91c3-9dbfdaf98757 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:25.083857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 775/1564] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:20:25.084830 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d5d4fa-4de9-4fb9-91c3-9dbfdaf98757 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.088671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 790/1565] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:20:25.089561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90d8a1-917f-4730-a0f8-445a186ff92d admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.101213 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90d8a1-917f-4730-a0f8-445a186ff92d admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.102309 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d5d4fa-4de9-4fb9-91c3-9dbfdaf98757 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.106269 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53184ff5-ce37-40ec-8be5-ab2e7184bb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.106871 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53184ff5-ce37-40ec-8be5-ab2e7184bb4c admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.107178 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53184ff5-ce37-40ec-8be5-ab2e7184bb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.107325 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43595430-61a5-4ef5-b2d9-2e499d7cd3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.107446 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53184ff5-ce37-40ec-8be5-ab2e7184bb4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.108043 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43595430-61a5-4ef5-b2d9-2e499d7cd3d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.108164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53184ff5-ce37-40ec-8be5-ab2e7184bb4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:20:25.108376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53184ff5-ce37-40ec-8be5-ab2e7184bb4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.108533 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43595430-61a5-4ef5-b2d9-2e499d7cd3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.108874 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43595430-61a5-4ef5-b2d9-2e499d7cd3d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.109603 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43595430-61a5-4ef5-b2d9-2e499d7cd3d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.109976 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53184ff5-ce37-40ec-8be5-ab2e7184bb4c admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:25.111726 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43595430-61a5-4ef5-b2d9-2e499d7cd3d8 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:25.112521 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:25.122737 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 791/1566] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:20:25.123665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53184ff5-ce37-40ec-8be5-ab2e7184bb4c admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.133352 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53184ff5-ce37-40ec-8be5-ab2e7184bb4c admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.138573 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad4974b-6d08-4a3c-8a93-4d34d7b40748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.139237 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4974b-6d08-4a3c-8a93-4d34d7b40748 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.139514 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4974b-6d08-4a3c-8a93-4d34d7b40748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.139791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4974b-6d08-4a3c-8a93-4d34d7b40748 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.140433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad4974b-6d08-4a3c-8a93-4d34d7b40748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.142021 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad4974b-6d08-4a3c-8a93-4d34d7b40748 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:25.150256 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 776/1567] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:20:25 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:25.150256 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43595430-61a5-4ef5-b2d9-2e499d7cd3d8 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.156618 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 792/1568] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:25.156618 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad4974b-6d08-4a3c-8a93-4d34d7b40748 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.164792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad4974b-6d08-4a3c-8a93-4d34d7b40748 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.210082 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43595430-61a5-4ef5-b2d9-2e499d7cd3d8 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.214902 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e30708-0d3f-4caa-990d-17bf0c3a4c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.215552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e30708-0d3f-4caa-990d-17bf0c3a4c54 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.215828 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e30708-0d3f-4caa-990d-17bf0c3a4c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.216092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e30708-0d3f-4caa-990d-17bf0c3a4c54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.216745 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e30708-0d3f-4caa-990d-17bf0c3a4c54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:20:25.217051 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e30708-0d3f-4caa-990d-17bf0c3a4c54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.218647 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e30708-0d3f-4caa-990d-17bf0c3a4c54 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:25.222468 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c55cee-3b08-49eb-941c-bed3a80365d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.223109 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c55cee-3b08-49eb-941c-bed3a80365d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.223390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c55cee-3b08-49eb-941c-bed3a80365d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.223659 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c55cee-3b08-49eb-941c-bed3a80365d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.224291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c55cee-3b08-49eb-941c-bed3a80365d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.226713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c55cee-3b08-49eb-941c-bed3a80365d2 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:25.231192 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 777/1569] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:20:25.231990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e30708-0d3f-4caa-990d-17bf0c3a4c54 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.243347 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e30708-0d3f-4caa-990d-17bf0c3a4c54 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.248041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd750d84-008a-4ce9-add5-beb099520b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.248646 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd750d84-008a-4ce9-add5-beb099520b9d admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.248922 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd750d84-008a-4ce9-add5-beb099520b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.249195 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd750d84-008a-4ce9-add5-beb099520b9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.249818 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd750d84-008a-4ce9-add5-beb099520b9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.253298 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 793/1570] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:20:25 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:25.253298 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c55cee-3b08-49eb-941c-bed3a80365d2 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.253541 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd750d84-008a-4ce9-add5-beb099520b9d admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:25.261080 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c55cee-3b08-49eb-941c-bed3a80365d2 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.265707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372de47c-5916-4ae6-9b33-5d054959b9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.266302 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372de47c-5916-4ae6-9b33-5d054959b9dd admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.266578 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372de47c-5916-4ae6-9b33-5d054959b9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.266844 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372de47c-5916-4ae6-9b33-5d054959b9dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.267497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372de47c-5916-4ae6-9b33-5d054959b9dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.268659 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: WARNING py.warnings [None req-372de47c-5916-4ae6-9b33-5d054959b9dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:20:25.268659 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: warnings.warn(msg) Aug 31 00:20:25.268659 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: Aug 31 00:20:25.270467 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372de47c-5916-4ae6-9b33-5d054959b9dd admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:25.279069 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 778/1571] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:20:25 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:25.279980 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd750d84-008a-4ce9-add5-beb099520b9d admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.304388 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 794/1572] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:20:25 2022] POST /identity/v3/projects => generated 393 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:25.304388 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372de47c-5916-4ae6-9b33-5d054959b9dd admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.347076 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd750d84-008a-4ce9-add5-beb099520b9d admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.355882 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:25.368671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372de47c-5916-4ae6-9b33-5d054959b9dd admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.373416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0cda28-386e-4c61-b40a-edc7be40f4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.374026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cda28-386e-4c61-b40a-edc7be40f4b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.374308 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cda28-386e-4c61-b40a-edc7be40f4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.374581 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cda28-386e-4c61-b40a-edc7be40f4b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.375239 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0cda28-386e-4c61-b40a-edc7be40f4b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.377150 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0cda28-386e-4c61-b40a-edc7be40f4b8 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:25.405101 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 795/1573] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:20:25 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:25.406402 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0cda28-386e-4c61-b40a-edc7be40f4b8 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.409093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e18102-d1e2-41b0-a4e6-eabb891abca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.409788 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e18102-d1e2-41b0-a4e6-eabb891abca5 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.410073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e18102-d1e2-41b0-a4e6-eabb891abca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.410347 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e18102-d1e2-41b0-a4e6-eabb891abca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.410981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e18102-d1e2-41b0-a4e6-eabb891abca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.412991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e18102-d1e2-41b0-a4e6-eabb891abca5 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:25.441287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 779/1574] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:20:25 2022] POST /identity/v3/projects => generated 377 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:25.442459 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e18102-d1e2-41b0-a4e6-eabb891abca5 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.453621 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:25.475208 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0cda28-386e-4c61-b40a-edc7be40f4b8 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.508457 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e18102-d1e2-41b0-a4e6-eabb891abca5 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.513655 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c3924-628e-43aa-8764-aaaada98921a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.514373 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c3924-628e-43aa-8764-aaaada98921a admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.514650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c3924-628e-43aa-8764-aaaada98921a admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.515180 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c3924-628e-43aa-8764-aaaada98921a admin admin] PATH_INFO: `/v3/projects/6ace4eb6c7da4bbd847646faf96ae9bf/users/c992fd6b00224edea70eb5a1ccc6d22b/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.529662 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0c3924-628e-43aa-8764-aaaada98921a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ace4eb6c7da4bbd847646faf96ae9bf, user_id=c992fd6b00224edea70eb5a1ccc6d22b, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.532418 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0c3924-628e-43aa-8764-aaaada98921a admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:25.532566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a44f2-6411-4c25-833a-d64dad31d581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.533155 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a44f2-6411-4c25-833a-d64dad31d581 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.533435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a44f2-6411-4c25-833a-d64dad31d581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.533709 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a44f2-6411-4c25-833a-d64dad31d581 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.534333 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a44f2-6411-4c25-833a-d64dad31d581 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.536302 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a44f2-6411-4c25-833a-d64dad31d581 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:25.545798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 780/1575] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:20:25 2022] PUT /identity/v3/projects/6ace4eb6c7da4bbd847646faf96ae9bf/users/c992fd6b00224edea70eb5a1ccc6d22b/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:25.546406 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c3924-628e-43aa-8764-aaaada98921a admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.574205 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 796/1576] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:20:25 2022] POST /identity/v3/users => generated 290 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:25.574205 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a44f2-6411-4c25-833a-d64dad31d581 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.581545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a44f2-6411-4c25-833a-d64dad31d581 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.611534 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c3924-628e-43aa-8764-aaaada98921a admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.617178 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29847e22-f152-4066-9901-6ccbbfcf13ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.617912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29847e22-f152-4066-9901-6ccbbfcf13ef admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.618188 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29847e22-f152-4066-9901-6ccbbfcf13ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.618458 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29847e22-f152-4066-9901-6ccbbfcf13ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.619144 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29847e22-f152-4066-9901-6ccbbfcf13ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.621015 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29847e22-f152-4066-9901-6ccbbfcf13ef admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:25.641749 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d97a2d-4228-475c-b980-408b22faa814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.642499 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d97a2d-4228-475c-b980-408b22faa814 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.642828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d97a2d-4228-475c-b980-408b22faa814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.643199 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d97a2d-4228-475c-b980-408b22faa814 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.643848 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d97a2d-4228-475c-b980-408b22faa814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.645448 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d97a2d-4228-475c-b980-408b22faa814 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:25.655605 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 797/1577] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:25.656182 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 781/1578] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:20:25 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:25.659992 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d97a2d-4228-475c-b980-408b22faa814 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.660160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29847e22-f152-4066-9901-6ccbbfcf13ef admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.666697 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:25.670307 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d97a2d-4228-475c-b980-408b22faa814 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.728696 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29847e22-f152-4066-9901-6ccbbfcf13ef admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.729871 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43d045-fbd9-4145-b185-7663f13050d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.730488 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43d045-fbd9-4145-b185-7663f13050d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.730775 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43d045-fbd9-4145-b185-7663f13050d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.731080 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43d045-fbd9-4145-b185-7663f13050d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.731720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb43d045-fbd9-4145-b185-7663f13050d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.733371 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765662b-f743-4257-8eff-ca7d3bbcd0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.733661 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb43d045-fbd9-4145-b185-7663f13050d6 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:25.733966 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765662b-f743-4257-8eff-ca7d3bbcd0a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.734242 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765662b-f743-4257-8eff-ca7d3bbcd0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.734507 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765662b-f743-4257-8eff-ca7d3bbcd0a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.735165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765662b-f743-4257-8eff-ca7d3bbcd0a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.737038 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765662b-f743-4257-8eff-ca7d3bbcd0a8 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:25.753377 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:25.759748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 798/1579] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:20:25 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:25.760933 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43d045-fbd9-4145-b185-7663f13050d6 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.761810 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 782/1580] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:20:25 2022] POST /identity/v3/users => generated 297 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:25.762796 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765662b-f743-4257-8eff-ca7d3bbcd0a8 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.772285 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765662b-f743-4257-8eff-ca7d3bbcd0a8 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.773991 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43d045-fbd9-4145-b185-7663f13050d6 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.777014 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6d439-2220-4a5a-9e69-f69eb12cc363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.777603 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6d439-2220-4a5a-9e69-f69eb12cc363 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.777883 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6d439-2220-4a5a-9e69-f69eb12cc363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.778152 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6d439-2220-4a5a-9e69-f69eb12cc363 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.778773 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6d439-2220-4a5a-9e69-f69eb12cc363 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.778892 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6998da9-4e6f-44ff-9786-52fc1a2216c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.779558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6998da9-4e6f-44ff-9786-52fc1a2216c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.779837 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6998da9-4e6f-44ff-9786-52fc1a2216c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.780241 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6998da9-4e6f-44ff-9786-52fc1a2216c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.780699 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6d439-2220-4a5a-9e69-f69eb12cc363 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:25.780873 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6998da9-4e6f-44ff-9786-52fc1a2216c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.782758 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6998da9-4e6f-44ff-9786-52fc1a2216c0 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:25.806344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 783/1581] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:20:25 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:25.807678 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ae245-05b0-4cb2-a063-e37b6921cc71 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.807998 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ae245-05b0-4cb2-a063-e37b6921cc71 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.808272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ae245-05b0-4cb2-a063-e37b6921cc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.812149 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 799/1582] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:20:25 2022] POST /identity/v3/users => generated 287 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:25.812149 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6998da9-4e6f-44ff-9786-52fc1a2216c0 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.820890 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6998da9-4e6f-44ff-9786-52fc1a2216c0 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.825955 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce08884d-cb72-46d7-9b61-f23cea886ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.826558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08884d-cb72-46d7-9b61-f23cea886ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.826838 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08884d-cb72-46d7-9b61-f23cea886ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.827146 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08884d-cb72-46d7-9b61-f23cea886ed9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.827788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce08884d-cb72-46d7-9b61-f23cea886ed9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.829441 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce08884d-cb72-46d7-9b61-f23cea886ed9 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:25.836831 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-2a6ae245-05b0-4cb2-a063-e37b6921cc71 None None] MFA Rules not processed for user `c992fd6b00224edea70eb5a1ccc6d22b`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:25.838035 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-2a6ae245-05b0-4cb2-a063-e37b6921cc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:25.840354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 800/1583] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:25.841157 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce08884d-cb72-46d7-9b61-f23cea886ed9 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.850961 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce08884d-cb72-46d7-9b61-f23cea886ed9 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.856088 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0405c7-e359-48a5-8ced-382b845a0e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.856685 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0405c7-e359-48a5-8ced-382b845a0e9e admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.856964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0405c7-e359-48a5-8ced-382b845a0e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.857231 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0405c7-e359-48a5-8ced-382b845a0e9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.857863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0405c7-e359-48a5-8ced-382b845a0e9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.859532 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0405c7-e359-48a5-8ced-382b845a0e9e admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:25.870430 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 801/1584] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:25.871058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0405c7-e359-48a5-8ced-382b845a0e9e admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.880321 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0405c7-e359-48a5-8ced-382b845a0e9e admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.885541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f91d6f-a83c-44af-91cb-968f70afb190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.886127 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f91d6f-a83c-44af-91cb-968f70afb190 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.886401 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f91d6f-a83c-44af-91cb-968f70afb190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.886668 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f91d6f-a83c-44af-91cb-968f70afb190 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.887324 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f91d6f-a83c-44af-91cb-968f70afb190 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.889240 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f91d6f-a83c-44af-91cb-968f70afb190 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:25.909052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 784/1585] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:25 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:25.910034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ae245-05b0-4cb2-a063-e37b6921cc71 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.914129 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 802/1586] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:20:25 2022] POST /identity/v3/users => generated 287 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:25.914129 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f91d6f-a83c-44af-91cb-968f70afb190 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.923104 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f91d6f-a83c-44af-91cb-968f70afb190 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.923343 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ae245-05b0-4cb2-a063-e37b6921cc71 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.927836 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca2372-a9bb-49f3-a2c8-b3486aca194a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.928012 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735a19b-b782-440f-8d1d-96acce931cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.928445 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca2372-a9bb-49f3-a2c8-b3486aca194a admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.929207 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735a19b-b782-440f-8d1d-96acce931cef admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.929418 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca2372-a9bb-49f3-a2c8-b3486aca194a admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.929531 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735a19b-b782-440f-8d1d-96acce931cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.929750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735a19b-b782-440f-8d1d-96acce931cef admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.929923 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca2372-a9bb-49f3-a2c8-b3486aca194a admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.930392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735a19b-b782-440f-8d1d-96acce931cef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.930606 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca2372-a9bb-49f3-a2c8-b3486aca194a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.932014 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735a19b-b782-440f-8d1d-96acce931cef admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:25.932239 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca2372-a9bb-49f3-a2c8-b3486aca194a admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:25.942214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 785/1587] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:25.943266 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 803/1588] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:25.943522 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735a19b-b782-440f-8d1d-96acce931cef admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.944541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca2372-a9bb-49f3-a2c8-b3486aca194a admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.955471 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca2372-a9bb-49f3-a2c8-b3486aca194a admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.955646 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735a19b-b782-440f-8d1d-96acce931cef admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.960160 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a1f50-2aa5-4c10-ad67-a99c85c470b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.960298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45284cdb-e903-46c6-90b4-d7dadbef68d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.960839 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a1f50-2aa5-4c10-ad67-a99c85c470b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.960958 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45284cdb-e903-46c6-90b4-d7dadbef68d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.961122 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a1f50-2aa5-4c10-ad67-a99c85c470b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.961228 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45284cdb-e903-46c6-90b4-d7dadbef68d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.961395 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a1f50-2aa5-4c10-ad67-a99c85c470b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.961500 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45284cdb-e903-46c6-90b4-d7dadbef68d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.962035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a1f50-2aa5-4c10-ad67-a99c85c470b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.962138 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45284cdb-e903-46c6-90b4-d7dadbef68d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.963659 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a1f50-2aa5-4c10-ad67-a99c85c470b9 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:25.963775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45284cdb-e903-46c6-90b4-d7dadbef68d5 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:25.977446 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 786/1589] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:25.977446 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45284cdb-e903-46c6-90b4-d7dadbef68d5 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.979217 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 804/1590] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:25.979999 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a1f50-2aa5-4c10-ad67-a99c85c470b9 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:25.986990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45284cdb-e903-46c6-90b4-d7dadbef68d5 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.989869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a1f50-2aa5-4c10-ad67-a99c85c470b9 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:25.993484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca9b599-45b2-4f3a-a506-3538da12516f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.994098 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9b599-45b2-4f3a-a506-3538da12516f admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.994382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9b599-45b2-4f3a-a506-3538da12516f admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.994659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9b599-45b2-4f3a-a506-3538da12516f admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.995028 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4ceb8-4e25-4756-b8cc-4c64b6c34ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:25.995340 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca9b599-45b2-4f3a-a506-3538da12516f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:25.995709 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4ceb8-4e25-4756-b8cc-4c64b6c34ee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:25.995990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4ceb8-4e25-4756-b8cc-4c64b6c34ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:25.996687 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4ceb8-4e25-4756-b8cc-4c64b6c34ee1 admin admin] PATH_INFO: `/v3/projects/885b952a63b34e1ba97215831cffc58e/users/6d1a15d59d5d476ca6e0ab5176d310cd/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:25.996956 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca9b599-45b2-4f3a-a506-3538da12516f admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:26.010156 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 787/1591] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:26.010998 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca9b599-45b2-4f3a-a506-3538da12516f admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:26.018160 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c4ceb8-4e25-4756-b8cc-4c64b6c34ee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=885b952a63b34e1ba97215831cffc58e, user_id=6d1a15d59d5d476ca6e0ab5176d310cd, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:26.019232 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: WARNING py.warnings [None req-68c4ceb8-4e25-4756-b8cc-4c64b6c34ee1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:20:26.019232 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: warnings.warn(msg) Aug 31 00:20:26.019232 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: Aug 31 00:20:26.021173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c4ceb8-4e25-4756-b8cc-4c64b6c34ee1 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:26.021736 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca9b599-45b2-4f3a-a506-3538da12516f admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:26.027828 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b126e0e-f1c9-4754-b95a-64d47d2d3a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:26.028520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b126e0e-f1c9-4754-b95a-64d47d2d3a3e admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:26.028802 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b126e0e-f1c9-4754-b95a-64d47d2d3a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:26.029108 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b126e0e-f1c9-4754-b95a-64d47d2d3a3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:26.029863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b126e0e-f1c9-4754-b95a-64d47d2d3a3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:26.031722 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b126e0e-f1c9-4754-b95a-64d47d2d3a3e admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:26.035060 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 805/1592] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:20:25 2022] PUT /identity/v3/projects/885b952a63b34e1ba97215831cffc58e/users/6d1a15d59d5d476ca6e0ab5176d310cd/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:26.035972 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4ceb8-4e25-4756-b8cc-4c64b6c34ee1 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:26.062702 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 788/1593] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:20:26 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:26.062702 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b126e0e-f1c9-4754-b95a-64d47d2d3a3e admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:26.076437 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:26.102189 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4ceb8-4e25-4756-b8cc-4c64b6c34ee1 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:26.128899 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b126e0e-f1c9-4754-b95a-64d47d2d3a3e admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:26.134978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7aecc-72d3-441e-969d-771c1c8001bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:26.135621 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7aecc-72d3-441e-969d-771c1c8001bb admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:26.135901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7aecc-72d3-441e-969d-771c1c8001bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:26.136175 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7aecc-72d3-441e-969d-771c1c8001bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:26.136824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7aecc-72d3-441e-969d-771c1c8001bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:26.138432 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7aecc-72d3-441e-969d-771c1c8001bb admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:26.148742 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 789/1594] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:26.149563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7aecc-72d3-441e-969d-771c1c8001bb admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:26.159397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7aecc-72d3-441e-969d-771c1c8001bb admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:26.163323 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eecefe-6fc7-46d3-901e-ca40ab4e9e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:26.164035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eecefe-6fc7-46d3-901e-ca40ab4e9e8b admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:26.164320 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eecefe-6fc7-46d3-901e-ca40ab4e9e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:26.164577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb2c28-5db3-474a-9c08-0548177252ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:26.164703 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eecefe-6fc7-46d3-901e-ca40ab4e9e8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:26.165195 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb2c28-5db3-474a-9c08-0548177252ef admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:26.165303 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eecefe-6fc7-46d3-901e-ca40ab4e9e8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:26.165498 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb2c28-5db3-474a-9c08-0548177252ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:26.165756 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb2c28-5db3-474a-9c08-0548177252ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:26.166389 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb2c28-5db3-474a-9c08-0548177252ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:26.166876 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eecefe-6fc7-46d3-901e-ca40ab4e9e8b admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:26.168076 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb2c28-5db3-474a-9c08-0548177252ef admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:26.172068 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:26.177405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 806/1595] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:26.182590 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 790/1596] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:26.182590 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb2c28-5db3-474a-9c08-0548177252ef admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:26.182806 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eecefe-6fc7-46d3-901e-ca40ab4e9e8b admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:26.191731 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb2c28-5db3-474a-9c08-0548177252ef admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:26.191886 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eecefe-6fc7-46d3-901e-ca40ab4e9e8b admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:26.197381 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc008a8f-d8c4-4d8a-a7e9-afb1ddaf33b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:26.197678 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011e011-ed80-4c5e-befc-e8626b3b3161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:26.198025 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc008a8f-d8c4-4d8a-a7e9-afb1ddaf33b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:26.198273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011e011-ed80-4c5e-befc-e8626b3b3161 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:26.198391 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc008a8f-d8c4-4d8a-a7e9-afb1ddaf33b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:26.198559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011e011-ed80-4c5e-befc-e8626b3b3161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:26.198676 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc008a8f-d8c4-4d8a-a7e9-afb1ddaf33b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:26.198844 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011e011-ed80-4c5e-befc-e8626b3b3161 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:26.199302 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc008a8f-d8c4-4d8a-a7e9-afb1ddaf33b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:26.199506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9011e011-ed80-4c5e-befc-e8626b3b3161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:26.200892 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc008a8f-d8c4-4d8a-a7e9-afb1ddaf33b0 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:26.201102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9011e011-ed80-4c5e-befc-e8626b3b3161 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:26.211803 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 807/1597] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:26.212764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 791/1598] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:26.212882 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc008a8f-d8c4-4d8a-a7e9-afb1ddaf33b0 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:26.213835 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1014cb5-db5a-4950-9910-7df51695bd7b None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:26.214117 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1014cb5-db5a-4950-9910-7df51695bd7b None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:26.214390 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1014cb5-db5a-4950-9910-7df51695bd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:26.223079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc008a8f-d8c4-4d8a-a7e9-afb1ddaf33b0 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:26.228113 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ceaac-2458-4884-bbdd-e45f7aaf2c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:26.228759 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ceaac-2458-4884-bbdd-e45f7aaf2c58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:26.229036 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ceaac-2458-4884-bbdd-e45f7aaf2c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:26.229300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ceaac-2458-4884-bbdd-e45f7aaf2c58 admin admin] PATH_INFO: `/v3/projects/78407f2ba91a45b2b1573e877aeadbb1/users/b8a88388dd8a45249e975fc92803778e/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:26.234041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-d1014cb5-db5a-4950-9910-7df51695bd7b None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:26.235277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-d1014cb5-db5a-4950-9910-7df51695bd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:26.243947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 792/1599] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:26.244615 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5ceaac-2458-4884-bbdd-e45f7aaf2c58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78407f2ba91a45b2b1573e877aeadbb1, user_id=b8a88388dd8a45249e975fc92803778e, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:26.244745 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1014cb5-db5a-4950-9910-7df51695bd7b None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:26.247328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5ceaac-2458-4884-bbdd-e45f7aaf2c58 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:26.254117 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1014cb5-db5a-4950-9910-7df51695bd7b None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:26.260939 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 808/1600] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:20:26 2022] PUT /identity/v3/projects/78407f2ba91a45b2b1573e877aeadbb1/users/b8a88388dd8a45249e975fc92803778e/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:26.261728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ceaac-2458-4884-bbdd-e45f7aaf2c58 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:26.313327 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e599cb2-9c4d-4793-8c46-165e46e6ec02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:26.314294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e599cb2-9c4d-4793-8c46-165e46e6ec02 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:26.314575 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e599cb2-9c4d-4793-8c46-165e46e6ec02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:26.314844 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e599cb2-9c4d-4793-8c46-165e46e6ec02 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:26.315839 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e599cb2-9c4d-4793-8c46-165e46e6ec02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:26.317738 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e599cb2-9c4d-4793-8c46-165e46e6ec02 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:26.334239 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ceaac-2458-4884-bbdd-e45f7aaf2c58 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:26.339432 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267bfdcf-9650-4148-9c24-cb197757fe54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:26.342591 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 793/1601] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:20:26 2022] POST /identity/v3/users => generated 309 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:26.342729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bfdcf-9650-4148-9c24-cb197757fe54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:26.342729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bfdcf-9650-4148-9c24-cb197757fe54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:26.342935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e599cb2-9c4d-4793-8c46-165e46e6ec02 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:26.343022 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bfdcf-9650-4148-9c24-cb197757fe54 admin admin] PATH_INFO: `/v3/projects/5694aa9e83804fe19311f97ed6af1152/users/7d7582f9cdd646f39404f7adc99ad9ac/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:26.349949 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e599cb2-9c4d-4793-8c46-165e46e6ec02 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:26.355620 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f5282-8397-4afa-a40d-a8f68ae38ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:26.356203 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267bfdcf-9650-4148-9c24-cb197757fe54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5694aa9e83804fe19311f97ed6af1152, user_id=7d7582f9cdd646f39404f7adc99ad9ac, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:26.356345 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f5282-8397-4afa-a40d-a8f68ae38ed0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:26.356537 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f5282-8397-4afa-a40d-a8f68ae38ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:26.356840 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f5282-8397-4afa-a40d-a8f68ae38ed0 admin admin] PATH_INFO: `/v3/projects/6a4d75f50b854a909607941db6debb87/users/3ff39cd0e766400f9d25e3ad465c4faf/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:26.358847 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267bfdcf-9650-4148-9c24-cb197757fe54 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:26.372399 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f5282-8397-4afa-a40d-a8f68ae38ed0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a4d75f50b854a909607941db6debb87, user_id=3ff39cd0e766400f9d25e3ad465c4faf, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:26.373281 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 809/1602] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:20:26 2022] PUT /identity/v3/projects/5694aa9e83804fe19311f97ed6af1152/users/7d7582f9cdd646f39404f7adc99ad9ac/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:26.374241 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267bfdcf-9650-4148-9c24-cb197757fe54 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:26.375107 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f5282-8397-4afa-a40d-a8f68ae38ed0 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:26.389770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 794/1603] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:20:26 2022] PUT /identity/v3/projects/6a4d75f50b854a909607941db6debb87/users/3ff39cd0e766400f9d25e3ad465c4faf/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:26.390536 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f5282-8397-4afa-a40d-a8f68ae38ed0 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:26.439094 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267bfdcf-9650-4148-9c24-cb197757fe54 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:26.442516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:26.460361 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f5282-8397-4afa-a40d-a8f68ae38ed0 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:26.465123 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb419e-a43d-4f74-bb16-401d927f4a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:26.465893 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb419e-a43d-4f74-bb16-401d927f4a73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:26.466200 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb419e-a43d-4f74-bb16-401d927f4a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:26.466486 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb419e-a43d-4f74-bb16-401d927f4a73 admin admin] PATH_INFO: `/v3/projects/788926197c5042b99931f959426cff47/users/d10b50451d4b48f0b2377626ed609de4/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:26.481564 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb419e-a43d-4f74-bb16-401d927f4a73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=788926197c5042b99931f959426cff47, user_id=d10b50451d4b48f0b2377626ed609de4, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:26.484246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb419e-a43d-4f74-bb16-401d927f4a73 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:26.489811 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:26.497989 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 795/1604] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:20:26 2022] PUT /identity/v3/projects/788926197c5042b99931f959426cff47/users/d10b50451d4b48f0b2377626ed609de4/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:26.498162 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa1627-4214-4d51-af20-0b6eaaa2c9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:26.498642 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa1627-4214-4d51-af20-0b6eaaa2c9a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:26.498907 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa1627-4214-4d51-af20-0b6eaaa2c9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:26.499539 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa1627-4214-4d51-af20-0b6eaaa2c9a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:26.500197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa1627-4214-4d51-af20-0b6eaaa2c9a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:26.502117 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa1627-4214-4d51-af20-0b6eaaa2c9a7 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:26.502345 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb419e-a43d-4f74-bb16-401d927f4a73 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:26.513969 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 810/1605] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:26.515114 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa1627-4214-4d51-af20-0b6eaaa2c9a7 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:26.517080 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-9cfa1627-4214-4d51-af20-0b6eaaa2c9a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:26.565662 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb419e-a43d-4f74-bb16-401d927f4a73 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:26.571381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c851d7-31d0-4e6a-b5de-ccb127039e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:26.572072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c851d7-31d0-4e6a-b5de-ccb127039e2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:26.572349 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c851d7-31d0-4e6a-b5de-ccb127039e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:26.572620 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c851d7-31d0-4e6a-b5de-ccb127039e2c admin admin] PATH_INFO: `/v3/projects/a71dcf1bec324a38bb5f120f295809e2/users/f6dc93317d5f473586dd2682f484c203/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:26.575309 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa1627-4214-4d51-af20-0b6eaaa2c9a7 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:26.587638 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c851d7-31d0-4e6a-b5de-ccb127039e2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a71dcf1bec324a38bb5f120f295809e2, user_id=f6dc93317d5f473586dd2682f484c203, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:26.590279 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c851d7-31d0-4e6a-b5de-ccb127039e2c admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:26.604093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 796/1606] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:20:26 2022] PUT /identity/v3/projects/a71dcf1bec324a38bb5f120f295809e2/users/f6dc93317d5f473586dd2682f484c203/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:26.605325 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ff6e7-062b-4355-a3e6-ea440d29e0c6 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:26.605614 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ff6e7-062b-4355-a3e6-ea440d29e0c6 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:26.605888 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ff6e7-062b-4355-a3e6-ea440d29e0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:26.626999 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d03cf-32b5-4be1-a062-1f1f29521df5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:26.627637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d03cf-32b5-4be1-a062-1f1f29521df5 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:26.627922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d03cf-32b5-4be1-a062-1f1f29521df5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:26.628203 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d03cf-32b5-4be1-a062-1f1f29521df5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:26.633147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-350ff6e7-062b-4355-a3e6-ea440d29e0c6 None None] MFA Rules not processed for user `b8a88388dd8a45249e975fc92803778e`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:26.634599 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-350ff6e7-062b-4355-a3e6-ea440d29e0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:26.638786 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d03cf-32b5-4be1-a062-1f1f29521df5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:26.640325 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d03cf-32b5-4be1-a062-1f1f29521df5 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:26.651386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 811/1607] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:26.652137 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d03cf-32b5-4be1-a062-1f1f29521df5 service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:26.662012 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d03cf-32b5-4be1-a062-1f1f29521df5 service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:26.713794 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 797/1608] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:26.718000 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c65602-7f44-4b69-8020-8d66b17b772c None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:26.718400 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c65602-7f44-4b69-8020-8d66b17b772c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:26.718723 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c65602-7f44-4b69-8020-8d66b17b772c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:26.730997 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa0090-0606-4c78-99d9-0b30b8accfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:26.731795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa0090-0606-4c78-99d9-0b30b8accfb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:26.732077 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa0090-0606-4c78-99d9-0b30b8accfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:26.732656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa0090-0606-4c78-99d9-0b30b8accfb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:26.733177 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa0090-0606-4c78-99d9-0b30b8accfb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:26.734996 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa0090-0606-4c78-99d9-0b30b8accfb2 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:26.739471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-12c65602-7f44-4b69-8020-8d66b17b772c None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:26.740838 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-12c65602-7f44-4b69-8020-8d66b17b772c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:26.747070 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 812/1609] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:26.751628 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685feaf-b08c-470a-9861-76c6a6697c9b None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:26.752064 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685feaf-b08c-470a-9861-76c6a6697c9b None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:26.752446 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685feaf-b08c-470a-9861-76c6a6697c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:26.753591 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 798/1610] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:26.755310 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94caa567-b587-42d4-a9ec-45d22c3ef683 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:26.756037 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94caa567-b587-42d4-a9ec-45d22c3ef683 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:26.756037 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94caa567-b587-42d4-a9ec-45d22c3ef683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:26.778584 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-94caa567-b587-42d4-a9ec-45d22c3ef683 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:26.779850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-94caa567-b587-42d4-a9ec-45d22c3ef683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:26.783481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-7685feaf-b08c-470a-9861-76c6a6697c9b None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:26.784736 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-7685feaf-b08c-470a-9861-76c6a6697c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:26.790304 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 799/1611] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:26.791174 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94caa567-b587-42d4-a9ec-45d22c3ef683 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:26.794192 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 813/1612] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:26.795508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812f85c-2599-4a89-a80e-357a80ec7c48 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:26.795792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812f85c-2599-4a89-a80e-357a80ec7c48 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:26.796066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812f85c-2599-4a89-a80e-357a80ec7c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:26.805662 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94caa567-b587-42d4-a9ec-45d22c3ef683 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:26.811694 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d77b4-6964-4bc3-804f-d5a0dcd3677d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:26.812416 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d77b4-6964-4bc3-804f-d5a0dcd3677d admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:26.812702 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d77b4-6964-4bc3-804f-d5a0dcd3677d admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:26.812979 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d77b4-6964-4bc3-804f-d5a0dcd3677d admin admin] PATH_INFO: `/v3/projects/6d33f302a33c427a8c2603ff5a6c1688/users/da22f994cb84433a8269ddc025727273/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:26.830298 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-1812f85c-2599-4a89-a80e-357a80ec7c48 None None] MFA Rules not processed for user `f6dc93317d5f473586dd2682f484c203`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:26.831801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-1812f85c-2599-4a89-a80e-357a80ec7c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:26.841573 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d77b4-6964-4bc3-804f-d5a0dcd3677d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d33f302a33c427a8c2603ff5a6c1688, user_id=da22f994cb84433a8269ddc025727273, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:26.845086 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d77b4-6964-4bc3-804f-d5a0dcd3677d admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:26.863887 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 800/1613] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:20:26 2022] PUT /identity/v3/projects/6d33f302a33c427a8c2603ff5a6c1688/users/da22f994cb84433a8269ddc025727273/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:26.865222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d77b4-6964-4bc3-804f-d5a0dcd3677d admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:26.877372 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:26.883354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:26.917800 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 814/1614] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:26.918826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812f85c-2599-4a89-a80e-357a80ec7c48 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:26.934542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812f85c-2599-4a89-a80e-357a80ec7c48 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:26.959603 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d77b4-6964-4bc3-804f-d5a0dcd3677d admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:26.966763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a820bcb-7591-480c-ae9c-1497a19c73c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:26.967486 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a820bcb-7591-480c-ae9c-1497a19c73c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:26.967770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a820bcb-7591-480c-ae9c-1497a19c73c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:26.968052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a820bcb-7591-480c-ae9c-1497a19c73c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:26.968737 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a820bcb-7591-480c-ae9c-1497a19c73c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:26.970952 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a820bcb-7591-480c-ae9c-1497a19c73c0 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:27.012986 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 801/1615] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:20:26 2022] POST /identity/v3/projects => generated 381 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:27.014350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a820bcb-7591-480c-ae9c-1497a19c73c0 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:27.045004 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0b32c-a5f1-44aa-8e13-9c8380dd39ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:27.045836 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0b32c-a5f1-44aa-8e13-9c8380dd39ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:27.046182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0b32c-a5f1-44aa-8e13-9c8380dd39ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:27.047494 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0b32c-a5f1-44aa-8e13-9c8380dd39ca admin admin] PATH_INFO: `/v3/projects/f9befe811fc14695851d518915574d4c/users/eb96862674ae41a4a77d7b663fed61b6/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:27.074920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0b32c-a5f1-44aa-8e13-9c8380dd39ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9befe811fc14695851d518915574d4c, user_id=eb96862674ae41a4a77d7b663fed61b6, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:27.080243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0b32c-a5f1-44aa-8e13-9c8380dd39ca admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:27.097776 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a820bcb-7591-480c-ae9c-1497a19c73c0 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:27.103111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 815/1616] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:20:26 2022] PUT /identity/v3/projects/f9befe811fc14695851d518915574d4c/users/eb96862674ae41a4a77d7b663fed61b6/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:27.104855 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0b32c-a5f1-44aa-8e13-9c8380dd39ca admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:27.170960 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536cc61-1d65-4ac9-a1ef-4659e6889c7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:27.171533 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536cc61-1d65-4ac9-a1ef-4659e6889c7b service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:27.171820 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536cc61-1d65-4ac9-a1ef-4659e6889c7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:27.172099 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536cc61-1d65-4ac9-a1ef-4659e6889c7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:27.182196 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0b32c-a5f1-44aa-8e13-9c8380dd39ca admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:27.186458 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a010a4-e3c5-4a53-b412-9791c8daa59b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:27.187227 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a010a4-e3c5-4a53-b412-9791c8daa59b service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:27.187554 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a010a4-e3c5-4a53-b412-9791c8daa59b service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:27.187782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a010a4-e3c5-4a53-b412-9791c8daa59b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:27.245521 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:27.250297 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c536cc61-1d65-4ac9-a1ef-4659e6889c7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:27.253481 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c536cc61-1d65-4ac9-a1ef-4659e6889c7b service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:27.258169 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a010a4-e3c5-4a53-b412-9791c8daa59b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:27.260246 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a010a4-e3c5-4a53-b412-9791c8daa59b service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:27.266940 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 802/1617] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:27.268066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536cc61-1d65-4ac9-a1ef-4659e6889c7b service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:27.271576 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 816/1618] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:27.273169 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19b132-30ea-483a-bf1a-daf65606f7c2 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:27.273711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19b132-30ea-483a-bf1a-daf65606f7c2 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:27.274293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19b132-30ea-483a-bf1a-daf65606f7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:27.277130 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536cc61-1d65-4ac9-a1ef-4659e6889c7b service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:27.280908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc433f8-625f-4514-a49a-4d25c9bf46b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:27.281370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:27.282582 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc433f8-625f-4514-a49a-4d25c9bf46b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:27.282870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc433f8-625f-4514-a49a-4d25c9bf46b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:27.283299 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc433f8-625f-4514-a49a-4d25c9bf46b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:27.295330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc433f8-625f-4514-a49a-4d25c9bf46b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:27.296876 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc433f8-625f-4514-a49a-4d25c9bf46b1 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:27.308259 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 803/1619] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:27.309339 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-8e19b132-30ea-483a-bf1a-daf65606f7c2 None None] MFA Rules not processed for user `da22f994cb84433a8269ddc025727273`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:27.309567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552889c-85f4-4a8d-aca1-8dbc6415acde None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:27.309716 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552889c-85f4-4a8d-aca1-8dbc6415acde None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:27.309939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552889c-85f4-4a8d-aca1-8dbc6415acde None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:27.310754 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-8e19b132-30ea-483a-bf1a-daf65606f7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:27.311337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 804/1620] 10.222.0.67 () {54 vars in 962 bytes} [Wed Aug 31 00:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:20:27.311864 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e552889c-85f4-4a8d-aca1-8dbc6415acde None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:27.326215 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e552889c-85f4-4a8d-aca1-8dbc6415acde None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:27.330801 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236038a2-08f5-4c8d-ad9d-957b45100586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:27.331511 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236038a2-08f5-4c8d-ad9d-957b45100586 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:27.331912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236038a2-08f5-4c8d-ad9d-957b45100586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:27.332214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236038a2-08f5-4c8d-ad9d-957b45100586 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:27.332941 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236038a2-08f5-4c8d-ad9d-957b45100586 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:27.335669 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236038a2-08f5-4c8d-ad9d-957b45100586 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:27.375021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 805/1621] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:20:27 2022] POST /identity/v3/users => generated 297 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:27.376066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236038a2-08f5-4c8d-ad9d-957b45100586 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:27.389567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236038a2-08f5-4c8d-ad9d-957b45100586 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:27.394839 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f668696-f362-42a8-9938-2aaa248f3310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:27.395847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f668696-f362-42a8-9938-2aaa248f3310 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:27.396411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f668696-f362-42a8-9938-2aaa248f3310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:27.396771 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f668696-f362-42a8-9938-2aaa248f3310 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:27.398449 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f668696-f362-42a8-9938-2aaa248f3310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:27.401007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f668696-f362-42a8-9938-2aaa248f3310 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:27.406653 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 817/1622] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:27.406653 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0627e9-7b94-432d-a22d-b452ef5408e9 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:27.406653 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0627e9-7b94-432d-a22d-b452ef5408e9 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:27.406653 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0627e9-7b94-432d-a22d-b452ef5408e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:27.427750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 806/1623] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:27.429235 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f668696-f362-42a8-9938-2aaa248f3310 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:27.443915 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f668696-f362-42a8-9938-2aaa248f3310 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:27.453677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b058e00-9651-4647-985e-842493787a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:27.454261 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b058e00-9651-4647-985e-842493787a2e admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:27.454890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b058e00-9651-4647-985e-842493787a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:27.455300 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b058e00-9651-4647-985e-842493787a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:27.456248 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b058e00-9651-4647-985e-842493787a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:27.458384 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b058e00-9651-4647-985e-842493787a2e admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:27.462661 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-3c0627e9-7b94-432d-a22d-b452ef5408e9 None None] MFA Rules not processed for user `3aa53ae5a30146f69d7d6eefe4290dba`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:27.463931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-3c0627e9-7b94-432d-a22d-b452ef5408e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:27.477303 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 807/1624] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:27.477303 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b058e00-9651-4647-985e-842493787a2e admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:27.486363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b058e00-9651-4647-985e-842493787a2e admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:27.492190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8a24e-6ec1-4020-8d68-2782b900df0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:27.492832 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8a24e-6ec1-4020-8d68-2782b900df0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:27.493180 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8a24e-6ec1-4020-8d68-2782b900df0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:27.493607 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8a24e-6ec1-4020-8d68-2782b900df0a admin admin] PATH_INFO: `/v3/projects/f9befe811fc14695851d518915574d4c/users/eb96862674ae41a4a77d7b663fed61b6/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:27.496995 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8a24e-6ec1-4020-8d68-2782b900df0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9befe811fc14695851d518915574d4c, user_id=eb96862674ae41a4a77d7b663fed61b6, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:27.500798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8a24e-6ec1-4020-8d68-2782b900df0a admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:27.515418 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 808/1625] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:20:27 2022] PUT /identity/v3/projects/f9befe811fc14695851d518915574d4c/users/eb96862674ae41a4a77d7b663fed61b6/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:27.517189 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d367ed2-e35e-4f12-b1de-c0bfada5b55a None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:27.517535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d367ed2-e35e-4f12-b1de-c0bfada5b55a None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:27.518026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d367ed2-e35e-4f12-b1de-c0bfada5b55a None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:27.519523 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 809/1626] 10.222.0.67 () {54 vars in 962 bytes} [Wed Aug 31 00:20:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:20:27.520515 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d367ed2-e35e-4f12-b1de-c0bfada5b55a None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:27.576167 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 818/1627] 10.222.0.67 () {58 vars in 1059 bytes} [Wed Aug 31 00:20:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:27.577877 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf39fa4-e440-4f5a-86dc-dd44dfc64411 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:27.578424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf39fa4-e440-4f5a-86dc-dd44dfc64411 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:27.578971 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf39fa4-e440-4f5a-86dc-dd44dfc64411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:27.643070 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-cbf39fa4-e440-4f5a-86dc-dd44dfc64411 None None] MFA Rules not processed for user `eb96862674ae41a4a77d7b663fed61b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:27.644243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-cbf39fa4-e440-4f5a-86dc-dd44dfc64411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:27.658443 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d367ed2-e35e-4f12-b1de-c0bfada5b55a None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:27.666504 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa423e90-36ae-451e-abde-cdf526588b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:27.667891 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa423e90-36ae-451e-abde-cdf526588b44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:27.668400 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa423e90-36ae-451e-abde-cdf526588b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:27.668814 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa423e90-36ae-451e-abde-cdf526588b44 admin admin] PATH_INFO: `/v3/projects/044bb7d6941242828dd02edcb2f793c9/users/fa672d3d8cf544fe814e391f05c1e221/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:27.698789 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa423e90-36ae-451e-abde-cdf526588b44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=044bb7d6941242828dd02edcb2f793c9, user_id=fa672d3d8cf544fe814e391f05c1e221, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:27.705639 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa423e90-36ae-451e-abde-cdf526588b44 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:27.729655 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 810/1628] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:20:27 2022] PUT /identity/v3/projects/044bb7d6941242828dd02edcb2f793c9/users/fa672d3d8cf544fe814e391f05c1e221/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:27.731460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e00c2c-c836-4b0d-ab28-1e6977d3ea92 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:27.731910 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e00c2c-c836-4b0d-ab28-1e6977d3ea92 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:27.732382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e00c2c-c836-4b0d-ab28-1e6977d3ea92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:27.771350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-43e00c2c-c836-4b0d-ab28-1e6977d3ea92 None None] MFA Rules not processed for user `3aa53ae5a30146f69d7d6eefe4290dba`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:27.774057 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-43e00c2c-c836-4b0d-ab28-1e6977d3ea92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:27.786350 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:27.809254 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 819/1629] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:27 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:27.810127 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf39fa4-e440-4f5a-86dc-dd44dfc64411 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:27.918096 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 811/1630] 10.222.0.67 () {58 vars in 1059 bytes} [Wed Aug 31 00:20:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:27.930013 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e00c2c-c836-4b0d-ab28-1e6977d3ea92 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:27.934336 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:27.966446 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf39fa4-e440-4f5a-86dc-dd44dfc64411 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:27.981681 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f8a39-fe35-4b72-964e-e049e0a3a2e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:27.982320 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f8a39-fe35-4b72-964e-e049e0a3a2e9 service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:27.982654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f8a39-fe35-4b72-964e-e049e0a3a2e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:27.983115 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f8a39-fe35-4b72-964e-e049e0a3a2e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:28.028584 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e00c2c-c836-4b0d-ab28-1e6977d3ea92 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:28.036047 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac81f0-4f0b-4ca8-82b8-39f5bb9c7459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:28.037327 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac81f0-4f0b-4ca8-82b8-39f5bb9c7459 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:28.037834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac81f0-4f0b-4ca8-82b8-39f5bb9c7459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:28.039325 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac81f0-4f0b-4ca8-82b8-39f5bb9c7459 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:28.040208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac81f0-4f0b-4ca8-82b8-39f5bb9c7459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:28.042788 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac81f0-4f0b-4ca8-82b8-39f5bb9c7459 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:28.057383 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 812/1631] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:28.058667 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac81f0-4f0b-4ca8-82b8-39f5bb9c7459 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:28.153341 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac81f0-4f0b-4ca8-82b8-39f5bb9c7459 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:28.155264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f8a39-fe35-4b72-964e-e049e0a3a2e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:28.157082 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f8a39-fe35-4b72-964e-e049e0a3a2e9 service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:28.161107 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef4cd36-b3cd-4b85-b6d2-a436a6267212 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:28.167661 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4cd36-b3cd-4b85-b6d2-a436a6267212 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:28.168473 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4cd36-b3cd-4b85-b6d2-a436a6267212 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:28.168473 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4cd36-b3cd-4b85-b6d2-a436a6267212 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:28.183701 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 820/1632] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:28.186779 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f8a39-fe35-4b72-964e-e049e0a3a2e9 service nova] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:28.207735 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f8a39-fe35-4b72-964e-e049e0a3a2e9 service nova] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:28.227325 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba34d6-fc7a-4824-a31b-1166e9e0e5c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:28.227874 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba34d6-fc7a-4824-a31b-1166e9e0e5c0 service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:28.228157 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba34d6-fc7a-4824-a31b-1166e9e0e5c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:28.228429 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba34d6-fc7a-4824-a31b-1166e9e0e5c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:28.269009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef4cd36-b3cd-4b85-b6d2-a436a6267212 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:28.270859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef4cd36-b3cd-4b85-b6d2-a436a6267212 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:28.294623 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 813/1633] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:28 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:28.296919 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef4cd36-b3cd-4b85-b6d2-a436a6267212 service cinder] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:28.316794 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef4cd36-b3cd-4b85-b6d2-a436a6267212 service cinder] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:28.324847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df65ba-06d4-4047-82e4-75e83cdd32df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:28.325566 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df65ba-06d4-4047-82e4-75e83cdd32df admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:28.326121 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df65ba-06d4-4047-82e4-75e83cdd32df admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:28.326927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df65ba-06d4-4047-82e4-75e83cdd32df admin admin] PATH_INFO: `/v3/projects/044bb7d6941242828dd02edcb2f793c9/users/fa672d3d8cf544fe814e391f05c1e221/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:28.329252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df65ba-06d4-4047-82e4-75e83cdd32df admin admin] RBAC: Authorizing `identity:create_grant(project_id=044bb7d6941242828dd02edcb2f793c9, user_id=fa672d3d8cf544fe814e391f05c1e221, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:28.332333 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df65ba-06d4-4047-82e4-75e83cdd32df admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:28.351415 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 814/1634] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:20:28 2022] PUT /identity/v3/projects/044bb7d6941242828dd02edcb2f793c9/users/fa672d3d8cf544fe814e391f05c1e221/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:28.364229 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471c023-b3b3-4500-a56b-5f457b692c2c None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:28.364557 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471c023-b3b3-4500-a56b-5f457b692c2c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:28.364795 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471c023-b3b3-4500-a56b-5f457b692c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:28.386256 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afba34d6-fc7a-4824-a31b-1166e9e0e5c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:28.399444 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afba34d6-fc7a-4824-a31b-1166e9e0e5c0 service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:28.416976 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-3471c023-b3b3-4500-a56b-5f457b692c2c None None] MFA Rules not processed for user `fa672d3d8cf544fe814e391f05c1e221`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:28.417135 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 821/1635] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:28.417544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-3471c023-b3b3-4500-a56b-5f457b692c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:28.494469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:28.645466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 815/1636] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:28 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:28.656713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba34d6-fc7a-4824-a31b-1166e9e0e5c0 service nova] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:28.672708 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba34d6-fc7a-4824-a31b-1166e9e0e5c0 service nova] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:28.673026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471c023-b3b3-4500-a56b-5f457b692c2c None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:28.744948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:28.797503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b09fea7-d7eb-4712-be4a-597ed65eb925 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:28.798130 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09fea7-d7eb-4712-be4a-597ed65eb925 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:28.798411 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09fea7-d7eb-4712-be4a-597ed65eb925 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:28.798682 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09fea7-d7eb-4712-be4a-597ed65eb925 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:28.801453 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471c023-b3b3-4500-a56b-5f457b692c2c None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:28.808774 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f27a229-dc07-4a5e-b49b-eb9458442234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:28.809490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f27a229-dc07-4a5e-b49b-eb9458442234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:28.810013 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f27a229-dc07-4a5e-b49b-eb9458442234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:28.810435 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f27a229-dc07-4a5e-b49b-eb9458442234 admin admin] PATH_INFO: `/v3/users/c992fd6b00224edea70eb5a1ccc6d22b` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:28.811656 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f27a229-dc07-4a5e-b49b-eb9458442234 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c992fd6b00224edea70eb5a1ccc6d22b)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:28.814153 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f27a229-dc07-4a5e-b49b-eb9458442234 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:28.820233 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b09fea7-d7eb-4712-be4a-597ed65eb925 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:28.821688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b09fea7-d7eb-4712-be4a-597ed65eb925 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:28.847721 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 822/1637] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:28.849216 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af4146-3959-4f7e-874a-22817a8b54df None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:28.849917 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af4146-3959-4f7e-874a-22817a8b54df None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:28.849917 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af4146-3959-4f7e-874a-22817a8b54df None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:28.852616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 823/1638] 10.222.0.67 () {54 vars in 912 bytes} [Wed Aug 31 00:20:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:20:28.859088 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db28cd7-dbd2-4d17-bee9-d626789821dc None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:28.859675 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db28cd7-dbd2-4d17-bee9-d626789821dc None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:28.860195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db28cd7-dbd2-4d17-bee9-d626789821dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:28.891865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-5db28cd7-dbd2-4d17-bee9-d626789821dc None None] MFA Rules not processed for user `3aa53ae5a30146f69d7d6eefe4290dba`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:28.893883 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-5db28cd7-dbd2-4d17-bee9-d626789821dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:28.912254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-1f27a229-dc07-4a5e-b49b-eb9458442234 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c992fd6b00224edea70eb5a1ccc6d22b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:28.941462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-1f27a229-dc07-4a5e-b49b-eb9458442234 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c992fd6b00224edea70eb5a1ccc6d22b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:28.953088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-1f27a229-dc07-4a5e-b49b-eb9458442234 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c992fd6b00224edea70eb5a1ccc6d22b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:28.964125 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-1f27a229-dc07-4a5e-b49b-eb9458442234 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c992fd6b00224edea70eb5a1ccc6d22b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:28.965856 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-1f27a229-dc07-4a5e-b49b-eb9458442234 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c992fd6b00224edea70eb5a1ccc6d22b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:28.981907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 816/1639] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:20:28 2022] DELETE /identity/v3/users/c992fd6b00224edea70eb5a1ccc6d22b => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:28.983855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97cd1e-f6cf-4963-b884-1fb244ac9c43 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:28.984357 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97cd1e-f6cf-4963-b884-1fb244ac9c43 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:28.986186 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97cd1e-f6cf-4963-b884-1fb244ac9c43 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:29.002354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 817/1640] 10.222.0.67 () {54 vars in 912 bytes} [Wed Aug 31 00:20:28 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:20:29.004286 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e38ed0-2db9-4e09-b6a7-a54a43fd8f81 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:29.005029 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e38ed0-2db9-4e09-b6a7-a54a43fd8f81 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:29.006083 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e38ed0-2db9-4e09-b6a7-a54a43fd8f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:29.040213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-47e38ed0-2db9-4e09-b6a7-a54a43fd8f81 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:29.041818 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-47e38ed0-2db9-4e09-b6a7-a54a43fd8f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:29.050778 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 824/1641] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 31 00:20:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:20:29.052621 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f67fa5-bd76-4c1a-9aa4-b39408bb9d52 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:29.053084 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f67fa5-bd76-4c1a-9aa4-b39408bb9d52 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:29.053366 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f67fa5-bd76-4c1a-9aa4-b39408bb9d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:29.083931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-31f67fa5-bd76-4c1a-9aa4-b39408bb9d52 None None] MFA Rules not processed for user `3aa53ae5a30146f69d7d6eefe4290dba`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:29.086888 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-31f67fa5-bd76-4c1a-9aa4-b39408bb9d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:29.088419 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:29.128783 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 818/1642] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:29.129935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e38ed0-2db9-4e09-b6a7-a54a43fd8f81 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:29.132610 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-47e38ed0-2db9-4e09-b6a7-a54a43fd8f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:29.197978 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 825/1643] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 31 00:20:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:29.198983 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f67fa5-bd76-4c1a-9aa4-b39408bb9d52 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:29.200517 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-31f67fa5-bd76-4c1a-9aa4-b39408bb9d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:29.207260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e38ed0-2db9-4e09-b6a7-a54a43fd8f81 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:29.211743 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac1d6b-774d-4e62-a12c-df3b919a5237 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:29.212373 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac1d6b-774d-4e62-a12c-df3b919a5237 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:29.212650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac1d6b-774d-4e62-a12c-df3b919a5237 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:29.212917 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac1d6b-774d-4e62-a12c-df3b919a5237 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:29.215320 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c6ac1d6b-774d-4e62-a12c-df3b919a5237 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:29.220363 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f67fa5-bd76-4c1a-9aa4-b39408bb9d52 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:29.225509 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9cc12-f1c3-41f1-ad3c-c7a3d21a46ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:29.226186 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9cc12-f1c3-41f1-ad3c-c7a3d21a46ba service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:29.226654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9cc12-f1c3-41f1-ad3c-c7a3d21a46ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:29.226939 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9cc12-f1c3-41f1-ad3c-c7a3d21a46ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:29.242151 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9cc12-f1c3-41f1-ad3c-c7a3d21a46ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:29.245047 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9cc12-f1c3-41f1-ad3c-c7a3d21a46ba service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:29.260977 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 826/1644] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:29.261845 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9cc12-f1c3-41f1-ad3c-c7a3d21a46ba service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:29.263300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-48b9cc12-f1c3-41f1-ad3c-c7a3d21a46ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:29.309746 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ac1d6b-774d-4e62-a12c-df3b919a5237 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:29.311237 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ac1d6b-774d-4e62-a12c-df3b919a5237 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:29.329613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 819/1645] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:29.334361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:29.346868 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9cc12-f1c3-41f1-ad3c-c7a3d21a46ba service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:29.351521 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a5829-3a1a-4543-8bc4-2d68fb2a6288 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:29.352323 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a5829-3a1a-4543-8bc4-2d68fb2a6288 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:29.352706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a5829-3a1a-4543-8bc4-2d68fb2a6288 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:29.353091 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a5829-3a1a-4543-8bc4-2d68fb2a6288 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:29.363958 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a5829-3a1a-4543-8bc4-2d68fb2a6288 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:29.365774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a5829-3a1a-4543-8bc4-2d68fb2a6288 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:29.386524 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 827/1646] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:29.394658 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac1d6b-774d-4e62-a12c-df3b919a5237 service cinder] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:29.419620 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac1d6b-774d-4e62-a12c-df3b919a5237 service cinder] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:29.423840 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0261b2b2-ce99-4f7e-8091-af1b470343c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:29.424931 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261b2b2-ce99-4f7e-8091-af1b470343c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:29.425216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261b2b2-ce99-4f7e-8091-af1b470343c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:29.425494 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261b2b2-ce99-4f7e-8091-af1b470343c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:29.437984 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0261b2b2-ce99-4f7e-8091-af1b470343c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:29.440972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0261b2b2-ce99-4f7e-8091-af1b470343c6 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:29.478272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 820/1647] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:30.083934 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a5829-3a1a-4543-8bc4-2d68fb2a6288 service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:30.086731 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-197a5829-3a1a-4543-8bc4-2d68fb2a6288 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:30.103510 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a5829-3a1a-4543-8bc4-2d68fb2a6288 service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:30.110660 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ee438-dfca-49df-b9e0-3e5659086bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:30.111547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ee438-dfca-49df-b9e0-3e5659086bed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:30.111738 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ee438-dfca-49df-b9e0-3e5659086bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:30.112189 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ee438-dfca-49df-b9e0-3e5659086bed admin admin] PATH_INFO: `/v3/users/f6dc93317d5f473586dd2682f484c203` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:30.113566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7ee438-dfca-49df-b9e0-3e5659086bed admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6dc93317d5f473586dd2682f484c203)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:30.114771 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: WARNING py.warnings [None req-5f7ee438-dfca-49df-b9e0-3e5659086bed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:20:30.114771 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: warnings.warn(msg) Aug 31 00:20:30.114771 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: Aug 31 00:20:30.116630 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7ee438-dfca-49df-b9e0-3e5659086bed admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:30.229464 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5f7ee438-dfca-49df-b9e0-3e5659086bed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6dc93317d5f473586dd2682f484c203'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:30.250885 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5f7ee438-dfca-49df-b9e0-3e5659086bed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6dc93317d5f473586dd2682f484c203'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:30.280575 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5f7ee438-dfca-49df-b9e0-3e5659086bed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6dc93317d5f473586dd2682f484c203'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:30.285337 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5f7ee438-dfca-49df-b9e0-3e5659086bed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6dc93317d5f473586dd2682f484c203'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:30.286038 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5f7ee438-dfca-49df-b9e0-3e5659086bed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6dc93317d5f473586dd2682f484c203'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:30.369415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 828/1648] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:20:30 2022] DELETE /identity/v3/users/f6dc93317d5f473586dd2682f484c203 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:30.371561 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61185d52-6ab2-4f1e-9024-ce64738e30e1 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:30.372128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61185d52-6ab2-4f1e-9024-ce64738e30e1 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:30.372227 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61185d52-6ab2-4f1e-9024-ce64738e30e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:30.397400 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-61185d52-6ab2-4f1e-9024-ce64738e30e1 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:30.398750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-61185d52-6ab2-4f1e-9024-ce64738e30e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:30.500125 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 821/1649] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:20:30.552148 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ee438-dfca-49df-b9e0-3e5659086bed admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:30.554177 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-5f7ee438-dfca-49df-b9e0-3e5659086bed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:30.682805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61185d52-6ab2-4f1e-9024-ce64738e30e1 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:30.684707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-61185d52-6ab2-4f1e-9024-ce64738e30e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:30.720459 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61185d52-6ab2-4f1e-9024-ce64738e30e1 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:30.782868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0da42b-7a23-463c-bdb4-af2339f1b26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:30.782868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0da42b-7a23-463c-bdb4-af2339f1b26a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:30.782868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0da42b-7a23-463c-bdb4-af2339f1b26a admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:30.782868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0da42b-7a23-463c-bdb4-af2339f1b26a admin admin] PATH_INFO: `/v3/projects/6ace4eb6c7da4bbd847646faf96ae9bf` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:30.787593 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ee438-dfca-49df-b9e0-3e5659086bed admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:30.788427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0da42b-7a23-463c-bdb4-af2339f1b26a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ace4eb6c7da4bbd847646faf96ae9bf)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:30.792126 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: WARNING py.warnings [None req-4e0da42b-7a23-463c-bdb4-af2339f1b26a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 00:20:30.792126 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: warnings.warn(msg) Aug 31 00:20:30.792126 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: Aug 31 00:20:30.803379 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0da42b-7a23-463c-bdb4-af2339f1b26a admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:30.809043 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a22bbe-894c-416f-a241-7c35a18bf99a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:30.812526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a22bbe-894c-416f-a241-7c35a18bf99a service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:30.815398 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a22bbe-894c-416f-a241-7c35a18bf99a service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:30.816209 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a22bbe-894c-416f-a241-7c35a18bf99a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:30.860574 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a22bbe-894c-416f-a241-7c35a18bf99a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:30.867893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a22bbe-894c-416f-a241-7c35a18bf99a service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:30.889651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:30.915956 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 829/1650] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:30.985061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-4e0da42b-7a23-463c-bdb4-af2339f1b26a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ace4eb6c7da4bbd847646faf96ae9bf', 'request_id': 'req-4e0da42b-7a23-463c-bdb4-af2339f1b26a'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:30.999881 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-4e0da42b-7a23-463c-bdb4-af2339f1b26a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ace4eb6c7da4bbd847646faf96ae9bf', 'request_id': 'req-4e0da42b-7a23-463c-bdb4-af2339f1b26a'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:31.020060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-4e0da42b-7a23-463c-bdb4-af2339f1b26a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ace4eb6c7da4bbd847646faf96ae9bf', 'request_id': 'req-4e0da42b-7a23-463c-bdb4-af2339f1b26a'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:31.069923 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-4e0da42b-7a23-463c-bdb4-af2339f1b26a admin admin] The token cache is being invalidate because project 6ace4eb6c7da4bbd847646faf96ae9bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:20:31.070356 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-4e0da42b-7a23-463c-bdb4-af2339f1b26a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:31.071134 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-4e0da42b-7a23-463c-bdb4-af2339f1b26a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:31.075610 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 822/1651] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:20:30 2022] DELETE /identity/v3/projects/6ace4eb6c7da4bbd847646faf96ae9bf => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:20:31.077889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a22bbe-894c-416f-a241-7c35a18bf99a service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:31.093868 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a22bbe-894c-416f-a241-7c35a18bf99a service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:31.261053 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d01df-23b8-46fd-b6af-c69e2802e2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:31.263533 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d01df-23b8-46fd-b6af-c69e2802e2a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:31.265146 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d01df-23b8-46fd-b6af-c69e2802e2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:31.265333 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d01df-23b8-46fd-b6af-c69e2802e2a5 admin admin] PATH_INFO: `/v3/users/eb96862674ae41a4a77d7b663fed61b6` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:31.267886 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d01df-23b8-46fd-b6af-c69e2802e2a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb96862674ae41a4a77d7b663fed61b6)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:31.269871 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d01df-23b8-46fd-b6af-c69e2802e2a5 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:31.279539 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:31.377864 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8c7d01df-23b8-46fd-b6af-c69e2802e2a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb96862674ae41a4a77d7b663fed61b6'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:31.393137 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8c7d01df-23b8-46fd-b6af-c69e2802e2a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb96862674ae41a4a77d7b663fed61b6'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:31.405381 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8c7d01df-23b8-46fd-b6af-c69e2802e2a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb96862674ae41a4a77d7b663fed61b6'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:31.407183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8c7d01df-23b8-46fd-b6af-c69e2802e2a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb96862674ae41a4a77d7b663fed61b6'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:31.408253 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8c7d01df-23b8-46fd-b6af-c69e2802e2a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb96862674ae41a4a77d7b663fed61b6'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:31.431554 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 830/1652] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:20:31 2022] DELETE /identity/v3/users/eb96862674ae41a4a77d7b663fed61b6 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:32.151811 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0da42b-7a23-463c-bdb4-af2339f1b26a admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:32.153884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-4e0da42b-7a23-463c-bdb4-af2339f1b26a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:32.176634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0da42b-7a23-463c-bdb4-af2339f1b26a admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:32.274880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb8269-1035-4799-ad83-d0eab6ee95e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:32.276176 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb8269-1035-4799-ad83-d0eab6ee95e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:32.277411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb8269-1035-4799-ad83-d0eab6ee95e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:32.277849 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb8269-1035-4799-ad83-d0eab6ee95e0 admin admin] PATH_INFO: `/v3/projects/f9befe811fc14695851d518915574d4c` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:32.279290 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb8269-1035-4799-ad83-d0eab6ee95e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9befe811fc14695851d518915574d4c)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:32.282572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb8269-1035-4799-ad83-d0eab6ee95e0 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:32.399929 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-15bb8269-1035-4799-ad83-d0eab6ee95e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9befe811fc14695851d518915574d4c', 'request_id': 'req-15bb8269-1035-4799-ad83-d0eab6ee95e0'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:32.415957 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-15bb8269-1035-4799-ad83-d0eab6ee95e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9befe811fc14695851d518915574d4c', 'request_id': 'req-15bb8269-1035-4799-ad83-d0eab6ee95e0'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:32.432873 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-15bb8269-1035-4799-ad83-d0eab6ee95e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9befe811fc14695851d518915574d4c', 'request_id': 'req-15bb8269-1035-4799-ad83-d0eab6ee95e0'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:32.474985 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-15bb8269-1035-4799-ad83-d0eab6ee95e0 admin admin] The token cache is being invalidate because project f9befe811fc14695851d518915574d4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:20:32.479188 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-15bb8269-1035-4799-ad83-d0eab6ee95e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:32.481906 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-15bb8269-1035-4799-ad83-d0eab6ee95e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:32.484114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 823/1653] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:20:32 2022] DELETE /identity/v3/projects/f9befe811fc14695851d518915574d4c => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:32.488833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03350d39-7db4-4b07-b96b-fde951882fcb None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:32.489352 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03350d39-7db4-4b07-b96b-fde951882fcb None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:32.489352 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03350d39-7db4-4b07-b96b-fde951882fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:32.528303 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-03350d39-7db4-4b07-b96b-fde951882fcb None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:32.530324 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-03350d39-7db4-4b07-b96b-fde951882fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:32.655280 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 831/1654] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:32.668804 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214d09b-7ed8-4626-b418-c6af4e804570 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:32.669354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214d09b-7ed8-4626-b418-c6af4e804570 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:32.670241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214d09b-7ed8-4626-b418-c6af4e804570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:32.713798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-3214d09b-7ed8-4626-b418-c6af4e804570 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:32.715071 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-3214d09b-7ed8-4626-b418-c6af4e804570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:32.743652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 824/1655] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:32.746706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03350d39-7db4-4b07-b96b-fde951882fcb None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:32.757459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03350d39-7db4-4b07-b96b-fde951882fcb None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:32.808837 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd50089-6e3c-4e1e-bfe8-724a051f0ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:32.809503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd50089-6e3c-4e1e-bfe8-724a051f0ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:32.809790 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd50089-6e3c-4e1e-bfe8-724a051f0ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:32.810071 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd50089-6e3c-4e1e-bfe8-724a051f0ed3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:32.810750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd50089-6e3c-4e1e-bfe8-724a051f0ed3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:20:32.811079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd50089-6e3c-4e1e-bfe8-724a051f0ed3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:32.813151 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd50089-6e3c-4e1e-bfe8-724a051f0ed3 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:32.877277 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 832/1656] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:20:32.879709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214d09b-7ed8-4626-b418-c6af4e804570 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:32.899352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214d09b-7ed8-4626-b418-c6af4e804570 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:32.911177 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe0675-ee06-462c-8697-7177fe4155d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:32.913612 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe0675-ee06-462c-8697-7177fe4155d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:32.916223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe0675-ee06-462c-8697-7177fe4155d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:32.916584 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe0675-ee06-462c-8697-7177fe4155d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:32.918402 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe0675-ee06-462c-8697-7177fe4155d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:32.924919 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe0675-ee06-462c-8697-7177fe4155d1 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:32.983984 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 825/1657] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:20:32 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:32.987093 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd50089-6e3c-4e1e-bfe8-724a051f0ed3 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:33.005593 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd50089-6e3c-4e1e-bfe8-724a051f0ed3 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:33.261597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee8eaa-c285-4b58-ae96-79d2af09f1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:33.262165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee8eaa-c285-4b58-ae96-79d2af09f1cc admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:33.262522 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee8eaa-c285-4b58-ae96-79d2af09f1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:33.262871 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee8eaa-c285-4b58-ae96-79d2af09f1cc admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:33.263632 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee8eaa-c285-4b58-ae96-79d2af09f1cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:33.267382 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee8eaa-c285-4b58-ae96-79d2af09f1cc admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:33.321682 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:33.331015 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 833/1658] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:20:32 2022] POST /identity/v3/users => generated 287 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:33.335029 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe0675-ee06-462c-8697-7177fe4155d1 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:33.349704 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe0675-ee06-462c-8697-7177fe4155d1 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:33.362084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512e8b9-c31c-4531-abc9-9956cacfed37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:33.363082 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512e8b9-c31c-4531-abc9-9956cacfed37 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:33.363603 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512e8b9-c31c-4531-abc9-9956cacfed37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:33.364078 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512e8b9-c31c-4531-abc9-9956cacfed37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:33.365614 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5512e8b9-c31c-4531-abc9-9956cacfed37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:33.369525 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5512e8b9-c31c-4531-abc9-9956cacfed37 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:33.371465 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:33.392414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 826/1659] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:33.397232 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee8eaa-c285-4b58-ae96-79d2af09f1cc admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:33.416084 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee8eaa-c285-4b58-ae96-79d2af09f1cc admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:33.431820 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc7c6b-e633-4587-8d0b-0d18ece826c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:33.433094 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc7c6b-e633-4587-8d0b-0d18ece826c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:33.433094 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc7c6b-e633-4587-8d0b-0d18ece826c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:33.433310 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc7c6b-e633-4587-8d0b-0d18ece826c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:33.433976 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc7c6b-e633-4587-8d0b-0d18ece826c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:33.435705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc7c6b-e633-4587-8d0b-0d18ece826c6 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:33.467972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512e8b9-c31c-4531-abc9-9956cacfed37 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:33.470822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-5512e8b9-c31c-4531-abc9-9956cacfed37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:33.477167 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 834/1660] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:33.483805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc7c6b-e633-4587-8d0b-0d18ece826c6 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:33.504511 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512e8b9-c31c-4531-abc9-9956cacfed37 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:33.527731 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91167e67-ce93-473d-ae1d-ce7268dce088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:33.528570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc7c6b-e633-4587-8d0b-0d18ece826c6 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:33.530542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91167e67-ce93-473d-ae1d-ce7268dce088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:33.531434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91167e67-ce93-473d-ae1d-ce7268dce088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:33.532240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91167e67-ce93-473d-ae1d-ce7268dce088 admin admin] PATH_INFO: `/v3/projects/a71dcf1bec324a38bb5f120f295809e2` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:33.534393 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91167e67-ce93-473d-ae1d-ce7268dce088 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a71dcf1bec324a38bb5f120f295809e2)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:33.541560 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91167e67-ce93-473d-ae1d-ce7268dce088 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:33.544715 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017c8a5-8bc6-422e-b757-eb781830f943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:33.547406 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017c8a5-8bc6-422e-b757-eb781830f943 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:33.548707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017c8a5-8bc6-422e-b757-eb781830f943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:33.550871 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017c8a5-8bc6-422e-b757-eb781830f943 admin admin] PATH_INFO: `/v3/projects/4b0b9ee8b63d4b7d9cfe7011aff85564/users/7e43bbfb758048c291a980455613f202/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:33.608621 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017c8a5-8bc6-422e-b757-eb781830f943 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b0b9ee8b63d4b7d9cfe7011aff85564, user_id=7e43bbfb758048c291a980455613f202, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:33.627129 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017c8a5-8bc6-422e-b757-eb781830f943 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:33.697475 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 835/1661] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:20:33 2022] PUT /identity/v3/projects/4b0b9ee8b63d4b7d9cfe7011aff85564/users/7e43bbfb758048c291a980455613f202/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:33.708237 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935a129-d81a-4f49-8019-b521f56b9237 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:33.708766 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935a129-d81a-4f49-8019-b521f56b9237 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:33.712264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935a129-d81a-4f49-8019-b521f56b9237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:33.748657 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-91167e67-ce93-473d-ae1d-ce7268dce088 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a71dcf1bec324a38bb5f120f295809e2', 'request_id': 'req-91167e67-ce93-473d-ae1d-ce7268dce088'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:33.771509 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-91167e67-ce93-473d-ae1d-ce7268dce088 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a71dcf1bec324a38bb5f120f295809e2', 'request_id': 'req-91167e67-ce93-473d-ae1d-ce7268dce088'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:33.795152 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-91167e67-ce93-473d-ae1d-ce7268dce088 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a71dcf1bec324a38bb5f120f295809e2', 'request_id': 'req-91167e67-ce93-473d-ae1d-ce7268dce088'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:33.809501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-2935a129-d81a-4f49-8019-b521f56b9237 None None] MFA Rules not processed for user `7e43bbfb758048c291a980455613f202`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:33.810761 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-2935a129-d81a-4f49-8019-b521f56b9237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:33.818861 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-91167e67-ce93-473d-ae1d-ce7268dce088 admin admin] The token cache is being invalidate because project a71dcf1bec324a38bb5f120f295809e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:20:33.822202 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-91167e67-ce93-473d-ae1d-ce7268dce088 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:33.823373 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-91167e67-ce93-473d-ae1d-ce7268dce088 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:33.826192 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 827/1662] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:20:33 2022] DELETE /identity/v3/projects/a71dcf1bec324a38bb5f120f295809e2 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:20:33.834204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1dace-070a-4e64-97f9-77d8f8d33308 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:33.835645 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1dace-070a-4e64-97f9-77d8f8d33308 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:33.836113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1dace-070a-4e64-97f9-77d8f8d33308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:33.884204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-37f1dace-070a-4e64-97f9-77d8f8d33308 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:33.887873 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-37f1dace-070a-4e64-97f9-77d8f8d33308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:33.974276 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 836/1663] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:20:33.979614 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935a129-d81a-4f49-8019-b521f56b9237 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:33.983442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-2935a129-d81a-4f49-8019-b521f56b9237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:34.087958 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 828/1664] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:34.092205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d70aa8-3f42-44a5-858e-a2a156623f35 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:34.092530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d70aa8-3f42-44a5-858e-a2a156623f35 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:34.092813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d70aa8-3f42-44a5-858e-a2a156623f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:34.145445 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-c6d70aa8-3f42-44a5-858e-a2a156623f35 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:34.149512 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c6d70aa8-3f42-44a5-858e-a2a156623f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:34.169159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935a129-d81a-4f49-8019-b521f56b9237 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:34.176101 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 829/1665] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:34.178826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6cbdd0-88dd-4369-a7bb-f4dddf4b52a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:34.179490 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6cbdd0-88dd-4369-a7bb-f4dddf4b52a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:34.179771 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6cbdd0-88dd-4369-a7bb-f4dddf4b52a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:34.180044 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6cbdd0-88dd-4369-a7bb-f4dddf4b52a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:34.180687 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6cbdd0-88dd-4369-a7bb-f4dddf4b52a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:34.180790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d70aa8-3f42-44a5-858e-a2a156623f35 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:34.183333 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6cbdd0-88dd-4369-a7bb-f4dddf4b52a4 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:34.198722 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d70aa8-3f42-44a5-858e-a2a156623f35 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:34.213844 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225e471-63d4-438e-96a1-f8e20ef97489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:34.214574 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225e471-63d4-438e-96a1-f8e20ef97489 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:34.215667 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225e471-63d4-438e-96a1-f8e20ef97489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:34.215958 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225e471-63d4-438e-96a1-f8e20ef97489 admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:34.219565 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225e471-63d4-438e-96a1-f8e20ef97489 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:20:34.219886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225e471-63d4-438e-96a1-f8e20ef97489 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:34.221852 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225e471-63d4-438e-96a1-f8e20ef97489 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:34.239763 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 837/1666] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:20:33 2022] POST /identity/v3/projects => generated 373 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:34.240850 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6cbdd0-88dd-4369-a7bb-f4dddf4b52a4 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:34.260443 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 830/1667] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:20:34.261985 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225e471-63d4-438e-96a1-f8e20ef97489 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:34.284856 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225e471-63d4-438e-96a1-f8e20ef97489 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:34.323321 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:34.377528 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6cbdd0-88dd-4369-a7bb-f4dddf4b52a4 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:34.391917 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bdea6-4e58-4746-8d57-529c83215dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:34.392562 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bdea6-4e58-4746-8d57-529c83215dca admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:34.392941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bdea6-4e58-4746-8d57-529c83215dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:34.393164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bdea6-4e58-4746-8d57-529c83215dca admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:34.393785 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3bdea6-4e58-4746-8d57-529c83215dca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:34.396522 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3bdea6-4e58-4746-8d57-529c83215dca admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:34.424149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b4684-62a1-48b7-b958-2db8c4ba12c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:34.426986 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b4684-62a1-48b7-b958-2db8c4ba12c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:34.426986 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b4684-62a1-48b7-b958-2db8c4ba12c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:34.426986 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b4684-62a1-48b7-b958-2db8c4ba12c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:34.428938 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5b4684-62a1-48b7-b958-2db8c4ba12c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:34.433660 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5b4684-62a1-48b7-b958-2db8c4ba12c3 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:34.448700 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:34.480835 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 838/1668] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:20:34 2022] POST /identity/v3/users => generated 293 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:20:34.481700 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bdea6-4e58-4746-8d57-529c83215dca admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:34.490448 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-1e3bdea6-4e58-4746-8d57-529c83215dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:34.516221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 831/1669] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:20:34 2022] POST /identity/v3/projects => generated 373 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:34.524484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b4684-62a1-48b7-b958-2db8c4ba12c3 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:34.525619 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bdea6-4e58-4746-8d57-529c83215dca admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:34.676040 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c8615-6094-4a1c-bd58-37cdd0c3444c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:34.677508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c8615-6094-4a1c-bd58-37cdd0c3444c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:34.677928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c8615-6094-4a1c-bd58-37cdd0c3444c admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:34.678472 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c8615-6094-4a1c-bd58-37cdd0c3444c admin admin] PATH_INFO: `/v3/users/da22f994cb84433a8269ddc025727273` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:34.684933 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c8615-6094-4a1c-bd58-37cdd0c3444c admin admin] RBAC: Authorizing `identity:delete_user(user_id=da22f994cb84433a8269ddc025727273)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:34.704221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c8615-6094-4a1c-bd58-37cdd0c3444c admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:34.711302 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b4684-62a1-48b7-b958-2db8c4ba12c3 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:34.747890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540ffbf-61d5-41e0-b8eb-0a865b867485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:34.749750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540ffbf-61d5-41e0-b8eb-0a865b867485 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:34.750110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540ffbf-61d5-41e0-b8eb-0a865b867485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:34.750398 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540ffbf-61d5-41e0-b8eb-0a865b867485 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:34.752224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540ffbf-61d5-41e0-b8eb-0a865b867485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:34.766737 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540ffbf-61d5-41e0-b8eb-0a865b867485 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:34.796605 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 832/1670] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:34.798033 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540ffbf-61d5-41e0-b8eb-0a865b867485 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:34.813692 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-fd7c8615-6094-4a1c-bd58-37cdd0c3444c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da22f994cb84433a8269ddc025727273'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:34.824424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540ffbf-61d5-41e0-b8eb-0a865b867485 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:34.828391 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-fd7c8615-6094-4a1c-bd58-37cdd0c3444c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da22f994cb84433a8269ddc025727273'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:34.841843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45e970-db52-49d9-8415-bbd271b2e35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:34.842476 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45e970-db52-49d9-8415-bbd271b2e35d admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:34.843363 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-fd7c8615-6094-4a1c-bd58-37cdd0c3444c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da22f994cb84433a8269ddc025727273'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:34.845902 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45e970-db52-49d9-8415-bbd271b2e35d admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:34.846953 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-fd7c8615-6094-4a1c-bd58-37cdd0c3444c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da22f994cb84433a8269ddc025727273'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:34.847114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45e970-db52-49d9-8415-bbd271b2e35d admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:34.847648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d45e970-db52-49d9-8415-bbd271b2e35d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:34.848171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-fd7c8615-6094-4a1c-bd58-37cdd0c3444c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da22f994cb84433a8269ddc025727273'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:34.853505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d45e970-db52-49d9-8415-bbd271b2e35d admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:34.896090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 839/1671] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:20:34 2022] DELETE /identity/v3/users/da22f994cb84433a8269ddc025727273 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:34.898064 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c8615-6094-4a1c-bd58-37cdd0c3444c admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:34.904034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 833/1672] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:20:34 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:34.904034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe412c85-f01d-41f7-bbb6-3b431e2f34b3 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:34.904381 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-fd7c8615-6094-4a1c-bd58-37cdd0c3444c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:34.905302 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe412c85-f01d-41f7-bbb6-3b431e2f34b3 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:34.905607 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe412c85-f01d-41f7-bbb6-3b431e2f34b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:34.964051 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-fe412c85-f01d-41f7-bbb6-3b431e2f34b3 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:34.966085 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-fe412c85-f01d-41f7-bbb6-3b431e2f34b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:35.026749 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:35.062951 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c8615-6094-4a1c-bd58-37cdd0c3444c admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:35.073653 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de85aa-b41f-4725-8255-54b451ca26e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:35.074354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de85aa-b41f-4725-8255-54b451ca26e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:35.074696 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de85aa-b41f-4725-8255-54b451ca26e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:35.074914 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de85aa-b41f-4725-8255-54b451ca26e4 admin admin] PATH_INFO: `/v3/projects/21a5a52e9fa043759ce05af5ef7f9fd8/users/bf0ecf872ce3405ab0b296c3568306cb/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:35.075981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 834/1673] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:35.076868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe412c85-f01d-41f7-bbb6-3b431e2f34b3 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:35.082388 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-fe412c85-f01d-41f7-bbb6-3b431e2f34b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:35.107726 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe412c85-f01d-41f7-bbb6-3b431e2f34b3 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:35.108924 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7de85aa-b41f-4725-8255-54b451ca26e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21a5a52e9fa043759ce05af5ef7f9fd8, user_id=bf0ecf872ce3405ab0b296c3568306cb, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:35.111648 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7de85aa-b41f-4725-8255-54b451ca26e4 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:35.113126 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f06d7-8faf-4f46-b84c-e94af41c71ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:35.113751 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f06d7-8faf-4f46-b84c-e94af41c71ce admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:35.114067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f06d7-8faf-4f46-b84c-e94af41c71ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:35.114300 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f06d7-8faf-4f46-b84c-e94af41c71ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:35.114950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73f06d7-8faf-4f46-b84c-e94af41c71ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:35.117570 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73f06d7-8faf-4f46-b84c-e94af41c71ce admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:35.133964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 835/1674] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:35.133964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f06d7-8faf-4f46-b84c-e94af41c71ce admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:35.136630 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-e73f06d7-8faf-4f46-b84c-e94af41c71ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:35.141930 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 840/1675] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:20:34 2022] PUT /identity/v3/projects/21a5a52e9fa043759ce05af5ef7f9fd8/users/bf0ecf872ce3405ab0b296c3568306cb/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:35.142958 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de85aa-b41f-4725-8255-54b451ca26e4 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:35.224725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f06d7-8faf-4f46-b84c-e94af41c71ce admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:35.242664 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:35.285117 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de85aa-b41f-4725-8255-54b451ca26e4 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:35.302170 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67504253-ec92-407c-8b2c-2263966c183d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:35.302920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67504253-ec92-407c-8b2c-2263966c183d admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:35.305330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67504253-ec92-407c-8b2c-2263966c183d admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:35.305712 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67504253-ec92-407c-8b2c-2263966c183d admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:35.308269 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67504253-ec92-407c-8b2c-2263966c183d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:35.311534 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67504253-ec92-407c-8b2c-2263966c183d admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:35.346357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 841/1676] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:35.348418 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67504253-ec92-407c-8b2c-2263966c183d admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:35.361280 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde4cc5-f791-477a-9eee-2bac9fd636fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:35.362359 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde4cc5-f791-477a-9eee-2bac9fd636fe service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:35.362891 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde4cc5-f791-477a-9eee-2bac9fd636fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:35.363417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde4cc5-f791-477a-9eee-2bac9fd636fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:35.379154 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde4cc5-f791-477a-9eee-2bac9fd636fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:35.385634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde4cc5-f791-477a-9eee-2bac9fd636fe service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:35.396865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67504253-ec92-407c-8b2c-2263966c183d admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:35.409371 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 836/1677] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:35.410531 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde4cc5-f791-477a-9eee-2bac9fd636fe service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:35.411805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647051b1-2adb-4eb0-bfea-67d5f9229822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:35.412469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647051b1-2adb-4eb0-bfea-67d5f9229822 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:35.412757 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647051b1-2adb-4eb0-bfea-67d5f9229822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:35.416460 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647051b1-2adb-4eb0-bfea-67d5f9229822 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:35.417363 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647051b1-2adb-4eb0-bfea-67d5f9229822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:35.422928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647051b1-2adb-4eb0-bfea-67d5f9229822 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:35.452842 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde4cc5-f791-477a-9eee-2bac9fd636fe service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:35.461371 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028c507-d847-478d-9ef4-9d2649b2e454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:35.462684 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028c507-d847-478d-9ef4-9d2649b2e454 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:35.462684 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028c507-d847-478d-9ef4-9d2649b2e454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:35.463156 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028c507-d847-478d-9ef4-9d2649b2e454 admin admin] PATH_INFO: `/v3/projects/5b50fab50ec442f08abb008f401b35f4/users/bd8f211823cd41d9a826e56a10e33e11/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:35.465688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 842/1678] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:35.470020 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647051b1-2adb-4eb0-bfea-67d5f9229822 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:35.496019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028c507-d847-478d-9ef4-9d2649b2e454 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b50fab50ec442f08abb008f401b35f4, user_id=bd8f211823cd41d9a826e56a10e33e11, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:35.515371 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647051b1-2adb-4eb0-bfea-67d5f9229822 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:35.518441 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028c507-d847-478d-9ef4-9d2649b2e454 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:35.542437 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0607a6-3893-4f1f-8403-064112fccf69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:35.543509 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0607a6-3893-4f1f-8403-064112fccf69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:35.545264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0607a6-3893-4f1f-8403-064112fccf69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:35.545721 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0607a6-3893-4f1f-8403-064112fccf69 admin admin] PATH_INFO: `/v3/projects/21a5a52e9fa043759ce05af5ef7f9fd8/users/bf0ecf872ce3405ab0b296c3568306cb/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:35.550579 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0607a6-3893-4f1f-8403-064112fccf69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21a5a52e9fa043759ce05af5ef7f9fd8, user_id=bf0ecf872ce3405ab0b296c3568306cb, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:35.555156 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0607a6-3893-4f1f-8403-064112fccf69 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:35.558603 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 837/1679] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:20:35 2022] PUT /identity/v3/projects/5b50fab50ec442f08abb008f401b35f4/users/bd8f211823cd41d9a826e56a10e33e11/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:35.573417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4e17a-1872-4049-8017-66f2f863e5bc None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:35.573708 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4e17a-1872-4049-8017-66f2f863e5bc None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:35.574342 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4e17a-1872-4049-8017-66f2f863e5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:35.581246 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 843/1680] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:20:35 2022] PUT /identity/v3/projects/21a5a52e9fa043759ce05af5ef7f9fd8/users/bf0ecf872ce3405ab0b296c3568306cb/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:35.596037 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e22c4-c1ee-477f-8ed4-55ae782cf516 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:35.596338 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e22c4-c1ee-477f-8ed4-55ae782cf516 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:35.596625 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e22c4-c1ee-477f-8ed4-55ae782cf516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:35.645095 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-3db4e17a-1872-4049-8017-66f2f863e5bc None None] MFA Rules not processed for user `bd8f211823cd41d9a826e56a10e33e11`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:35.646402 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-3db4e17a-1872-4049-8017-66f2f863e5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:35.676809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-f96e22c4-c1ee-477f-8ed4-55ae782cf516 None None] MFA Rules not processed for user `bf0ecf872ce3405ab0b296c3568306cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:35.679318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f96e22c4-c1ee-477f-8ed4-55ae782cf516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:35.867909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 838/1681] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:35.871024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4e17a-1872-4049-8017-66f2f863e5bc None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:35.957443 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 844/1682] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:35.976851 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e22c4-c1ee-477f-8ed4-55ae782cf516 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:35.979082 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f96e22c4-c1ee-477f-8ed4-55ae782cf516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:36.090224 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:36.124143 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4e17a-1872-4049-8017-66f2f863e5bc None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:36.160807 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c219d47-cb38-44b1-88fb-710cafb2a38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:36.161467 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c219d47-cb38-44b1-88fb-710cafb2a38a admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:36.164472 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c219d47-cb38-44b1-88fb-710cafb2a38a admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:36.164793 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c219d47-cb38-44b1-88fb-710cafb2a38a admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:36.165485 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c219d47-cb38-44b1-88fb-710cafb2a38a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:36.166685 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e22c4-c1ee-477f-8ed4-55ae782cf516 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:36.181261 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c219d47-cb38-44b1-88fb-710cafb2a38a admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:36.182544 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa909047-914d-4bb8-9b34-51f932a4fd69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:36.182544 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa909047-914d-4bb8-9b34-51f932a4fd69 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:36.182731 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa909047-914d-4bb8-9b34-51f932a4fd69 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:36.182954 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa909047-914d-4bb8-9b34-51f932a4fd69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:36.207378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa909047-914d-4bb8-9b34-51f932a4fd69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:36.208881 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa909047-914d-4bb8-9b34-51f932a4fd69 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:36.223245 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 839/1683] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:20:35 2022] POST /identity/v3/projects => generated 373 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:36.226989 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c219d47-cb38-44b1-88fb-710cafb2a38a admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:36.236577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:36.244944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 845/1684] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:36.359609 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c219d47-cb38-44b1-88fb-710cafb2a38a admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:36.367380 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4e21f-9460-4a27-af91-892b1f1aeff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:36.368657 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4e21f-9460-4a27-af91-892b1f1aeff6 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:36.369209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4e21f-9460-4a27-af91-892b1f1aeff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:36.369828 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4e21f-9460-4a27-af91-892b1f1aeff6 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:36.371220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4e21f-9460-4a27-af91-892b1f1aeff6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:36.376589 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4e21f-9460-4a27-af91-892b1f1aeff6 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:36.406874 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 840/1685] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:20:36 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:36.410374 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa909047-914d-4bb8-9b34-51f932a4fd69 service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:36.424332 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa909047-914d-4bb8-9b34-51f932a4fd69 service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:36.448309 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5f1b4-2556-4d68-b62a-8e30fd7280d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:36.448953 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5f1b4-2556-4d68-b62a-8e30fd7280d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:36.449237 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5f1b4-2556-4d68-b62a-8e30fd7280d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:36.449515 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5f1b4-2556-4d68-b62a-8e30fd7280d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:36.450181 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca5f1b4-2556-4d68-b62a-8e30fd7280d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:36.451850 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca5f1b4-2556-4d68-b62a-8e30fd7280d8 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:36.495407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 846/1686] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:36.499576 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4e21f-9460-4a27-af91-892b1f1aeff6 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:36.522046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4e21f-9460-4a27-af91-892b1f1aeff6 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:36.541180 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd619b-f2ed-419f-85b2-b7ee1e7a7843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:36.541880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd619b-f2ed-419f-85b2-b7ee1e7a7843 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:36.542165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd619b-f2ed-419f-85b2-b7ee1e7a7843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:36.542441 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd619b-f2ed-419f-85b2-b7ee1e7a7843 admin admin] PATH_INFO: `/v3/projects/17bcc1d36e49400593dc98619efe7ac2/users/367150656f694352bbc0ffa8aba6a5eb/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:36.603640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd619b-f2ed-419f-85b2-b7ee1e7a7843 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17bcc1d36e49400593dc98619efe7ac2, user_id=367150656f694352bbc0ffa8aba6a5eb, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:36.609619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd619b-f2ed-419f-85b2-b7ee1e7a7843 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:36.646998 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 841/1687] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:20:36 2022] PUT /identity/v3/projects/17bcc1d36e49400593dc98619efe7ac2/users/367150656f694352bbc0ffa8aba6a5eb/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:36.651462 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5f1b4-2556-4d68-b62a-8e30fd7280d8 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:36.668169 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5f1b4-2556-4d68-b62a-8e30fd7280d8 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:36.810026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff061be-bf47-424f-bf5b-96393fbe3abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:36.810596 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff061be-bf47-424f-bf5b-96393fbe3abd admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:36.811310 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff061be-bf47-424f-bf5b-96393fbe3abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:36.811310 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff061be-bf47-424f-bf5b-96393fbe3abd admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:36.811757 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff061be-bf47-424f-bf5b-96393fbe3abd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:36.813411 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff061be-bf47-424f-bf5b-96393fbe3abd admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:36.837336 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd619b-f2ed-419f-85b2-b7ee1e7a7843 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:36.848998 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 847/1688] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:36.853168 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd619b-f2ed-419f-85b2-b7ee1e7a7843 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:36.871977 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be7471-0e06-4763-a718-318b8a0fc320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:36.872824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be7471-0e06-4763-a718-318b8a0fc320 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:36.873161 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be7471-0e06-4763-a718-318b8a0fc320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:36.873469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be7471-0e06-4763-a718-318b8a0fc320 admin admin] PATH_INFO: `/v3/projects/17bcc1d36e49400593dc98619efe7ac2/users/367150656f694352bbc0ffa8aba6a5eb/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:36.886445 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be7471-0e06-4763-a718-318b8a0fc320 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17bcc1d36e49400593dc98619efe7ac2, user_id=367150656f694352bbc0ffa8aba6a5eb, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:36.887406 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be7471-0e06-4763-a718-318b8a0fc320 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:36.915598 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 842/1689] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:20:36 2022] PUT /identity/v3/projects/17bcc1d36e49400593dc98619efe7ac2/users/367150656f694352bbc0ffa8aba6a5eb/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:36.925745 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1db57-4166-45e8-adcd-44f5f3109615 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:36.926283 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1db57-4166-45e8-adcd-44f5f3109615 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:36.926843 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1db57-4166-45e8-adcd-44f5f3109615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:37.013703 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-d3f1db57-4166-45e8-adcd-44f5f3109615 None None] MFA Rules not processed for user `367150656f694352bbc0ffa8aba6a5eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:37.015619 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-d3f1db57-4166-45e8-adcd-44f5f3109615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:37.168033 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be7471-0e06-4763-a718-318b8a0fc320 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:37.304479 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 848/1690] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:37.324213 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1db57-4166-45e8-adcd-44f5f3109615 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:37.400975 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be7471-0e06-4763-a718-318b8a0fc320 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:37.412591 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22f1b9-f611-4182-b096-2e38c4713da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:37.413637 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22f1b9-f611-4182-b096-2e38c4713da7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:37.414860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22f1b9-f611-4182-b096-2e38c4713da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:37.415372 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22f1b9-f611-4182-b096-2e38c4713da7 admin admin] PATH_INFO: `/v3/users/7e43bbfb758048c291a980455613f202` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:37.417843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22f1b9-f611-4182-b096-2e38c4713da7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e43bbfb758048c291a980455613f202)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:37.422868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22f1b9-f611-4182-b096-2e38c4713da7 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:37.472621 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:37.490631 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1db57-4166-45e8-adcd-44f5f3109615 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:37.501419 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce79ec4-a7c1-4ab7-a5ee-8dd725d1ecf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:37.503308 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce79ec4-a7c1-4ab7-a5ee-8dd725d1ecf0 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:37.504236 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce79ec4-a7c1-4ab7-a5ee-8dd725d1ecf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:37.506337 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce79ec4-a7c1-4ab7-a5ee-8dd725d1ecf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:37.527318 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9b22f1b9-f611-4182-b096-2e38c4713da7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e43bbfb758048c291a980455613f202'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:37.527932 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce79ec4-a7c1-4ab7-a5ee-8dd725d1ecf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:37.531557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce79ec4-a7c1-4ab7-a5ee-8dd725d1ecf0 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:37.538712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:37.556131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9b22f1b9-f611-4182-b096-2e38c4713da7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e43bbfb758048c291a980455613f202'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:37.563213 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 849/1691] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:37 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:20:37.572601 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9b22f1b9-f611-4182-b096-2e38c4713da7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e43bbfb758048c291a980455613f202'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:37.587651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9b22f1b9-f611-4182-b096-2e38c4713da7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e43bbfb758048c291a980455613f202'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:37.588445 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9b22f1b9-f611-4182-b096-2e38c4713da7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e43bbfb758048c291a980455613f202'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:37.619237 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 843/1692] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:20:37 2022] DELETE /identity/v3/users/7e43bbfb758048c291a980455613f202 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:20:37.619768 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9b2f1-6e58-4bdb-981a-2a51ef5f1a9b None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:37.620061 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9b2f1-6e58-4bdb-981a-2a51ef5f1a9b None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:37.620339 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9b2f1-6e58-4bdb-981a-2a51ef5f1a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:37.662443 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-b4c9b2f1-6e58-4bdb-981a-2a51ef5f1a9b None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:37.665147 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b4c9b2f1-6e58-4bdb-981a-2a51ef5f1a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:37.729934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22f1b9-f611-4182-b096-2e38c4713da7 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:37.732228 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-9b22f1b9-f611-4182-b096-2e38c4713da7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:37.842319 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 850/1693] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:37.860871 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22f1b9-f611-4182-b096-2e38c4713da7 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:37.868073 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9b2f1-6e58-4bdb-981a-2a51ef5f1a9b None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:37.869084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e35e13-71df-4d80-89ae-cd00f202c086 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:37.870330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e35e13-71df-4d80-89ae-cd00f202c086 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:37.870944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e35e13-71df-4d80-89ae-cd00f202c086 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:37.871093 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b4c9b2f1-6e58-4bdb-981a-2a51ef5f1a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:37.871888 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e35e13-71df-4d80-89ae-cd00f202c086 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:37.878733 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f7e35e13-71df-4d80-89ae-cd00f202c086 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:38.000393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9b2f1-6e58-4bdb-981a-2a51ef5f1a9b None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:38.005269 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064fdd5f-9bdf-40f0-b09b-ab363670b470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:38.005992 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fdd5f-9bdf-40f0-b09b-ab363670b470 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:38.006877 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fdd5f-9bdf-40f0-b09b-ab363670b470 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:38.007552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fdd5f-9bdf-40f0-b09b-ab363670b470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:38.030533 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064fdd5f-9bdf-40f0-b09b-ab363670b470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:38.033976 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064fdd5f-9bdf-40f0-b09b-ab363670b470 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:38.039127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e35e13-71df-4d80-89ae-cd00f202c086 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:38.040713 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e35e13-71df-4d80-89ae-cd00f202c086 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:38.051583 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 844/1694] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:38.074758 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 851/1695] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:38.191691 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e35e13-71df-4d80-89ae-cd00f202c086 service cinder] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:38.193894 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f7e35e13-71df-4d80-89ae-cd00f202c086 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:38.219251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e35e13-71df-4d80-89ae-cd00f202c086 service cinder] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:38.261141 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8c711-32e4-4224-a189-57b8f1230cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:38.263624 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8c711-32e4-4224-a189-57b8f1230cc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:38.264484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8c711-32e4-4224-a189-57b8f1230cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:38.271614 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8c711-32e4-4224-a189-57b8f1230cc2 admin admin] PATH_INFO: `/v3/projects/6d33f302a33c427a8c2603ff5a6c1688` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:38.274148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f8c711-32e4-4224-a189-57b8f1230cc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d33f302a33c427a8c2603ff5a6c1688)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:38.276499 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f8c711-32e4-4224-a189-57b8f1230cc2 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:38.355188 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b5f8c711-32e4-4224-a189-57b8f1230cc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d33f302a33c427a8c2603ff5a6c1688', 'request_id': 'req-b5f8c711-32e4-4224-a189-57b8f1230cc2'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:38.367277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b5f8c711-32e4-4224-a189-57b8f1230cc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d33f302a33c427a8c2603ff5a6c1688', 'request_id': 'req-b5f8c711-32e4-4224-a189-57b8f1230cc2'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:38.382105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b5f8c711-32e4-4224-a189-57b8f1230cc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d33f302a33c427a8c2603ff5a6c1688', 'request_id': 'req-b5f8c711-32e4-4224-a189-57b8f1230cc2'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:38.413770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b5f8c711-32e4-4224-a189-57b8f1230cc2 admin admin] The token cache is being invalidate because project 6d33f302a33c427a8c2603ff5a6c1688 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:20:38.414119 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b5f8c711-32e4-4224-a189-57b8f1230cc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:38.421024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b5f8c711-32e4-4224-a189-57b8f1230cc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:38.423238 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 845/1696] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:20:38 2022] DELETE /identity/v3/projects/6d33f302a33c427a8c2603ff5a6c1688 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:38.427943 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337d1a7-d29a-4328-9c64-863d261db371 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:38.428235 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337d1a7-d29a-4328-9c64-863d261db371 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:38.428559 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337d1a7-d29a-4328-9c64-863d261db371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:38.478054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-9337d1a7-d29a-4328-9c64-863d261db371 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:38.479421 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-9337d1a7-d29a-4328-9c64-863d261db371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:38.682116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 852/1697] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:38.690344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b903228-f3e5-4698-88fa-7924b4f997bb None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:38.691179 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b903228-f3e5-4698-88fa-7924b4f997bb None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:38.691646 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b903228-f3e5-4698-88fa-7924b4f997bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:38.725918 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-0b903228-f3e5-4698-88fa-7924b4f997bb None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:38.727921 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-0b903228-f3e5-4698-88fa-7924b4f997bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:38.747362 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 846/1698] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:20:38.748356 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337d1a7-d29a-4328-9c64-863d261db371 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:38.764103 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337d1a7-d29a-4328-9c64-863d261db371 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:38.782413 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d44170-e4d2-4801-9198-f4eff405d4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:38.783085 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d44170-e4d2-4801-9198-f4eff405d4c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:38.783376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d44170-e4d2-4801-9198-f4eff405d4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:38.783649 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d44170-e4d2-4801-9198-f4eff405d4c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:38.784320 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d44170-e4d2-4801-9198-f4eff405d4c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:20:38.785941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d44170-e4d2-4801-9198-f4eff405d4c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:38.791739 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d44170-e4d2-4801-9198-f4eff405d4c3 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:38.829168 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 853/1699] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:20:38.834904 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b903228-f3e5-4698-88fa-7924b4f997bb None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:38.843497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:38.855209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b903228-f3e5-4698-88fa-7924b4f997bb None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:38.870363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702b1c3-c8d0-4171-8386-0f6f83938a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:38.871134 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702b1c3-c8d0-4171-8386-0f6f83938a1d admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:38.871413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702b1c3-c8d0-4171-8386-0f6f83938a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:38.871713 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702b1c3-c8d0-4171-8386-0f6f83938a1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:38.872374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e702b1c3-c8d0-4171-8386-0f6f83938a1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:38.875713 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e702b1c3-c8d0-4171-8386-0f6f83938a1d admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:38.952936 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d44170-e4d2-4801-9198-f4eff405d4c3 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:38.953723 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 847/1700] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:20:38 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:38.968959 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d44170-e4d2-4801-9198-f4eff405d4c3 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:38.987888 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:39.069013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c9dda-2f18-4912-950a-19ad4169dd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:39.070765 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c9dda-2f18-4912-950a-19ad4169dd0c admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:39.071263 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c9dda-2f18-4912-950a-19ad4169dd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:39.071558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c9dda-2f18-4912-950a-19ad4169dd0c admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:39.072374 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c9dda-2f18-4912-950a-19ad4169dd0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:39.074593 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c9dda-2f18-4912-950a-19ad4169dd0c admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:39.114350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702b1c3-c8d0-4171-8386-0f6f83938a1d admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:39.135235 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 854/1701] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:20:38 2022] POST /identity/v3/users => generated 283 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:39.156354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702b1c3-c8d0-4171-8386-0f6f83938a1d admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:39.163137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc209a-e9e9-4f42-b9d3-0b94bf528223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:39.163862 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc209a-e9e9-4f42-b9d3-0b94bf528223 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:39.164483 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc209a-e9e9-4f42-b9d3-0b94bf528223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:39.164824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc209a-e9e9-4f42-b9d3-0b94bf528223 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:39.165994 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc209a-e9e9-4f42-b9d3-0b94bf528223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:39.167867 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc209a-e9e9-4f42-b9d3-0b94bf528223 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:39.186319 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 848/1702] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:39.186524 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c9dda-2f18-4912-950a-19ad4169dd0c admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:39.211935 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c9dda-2f18-4912-950a-19ad4169dd0c admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:39.221323 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63293b6a-9796-4646-b2f6-070b4cd0c35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:39.222500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63293b6a-9796-4646-b2f6-070b4cd0c35e admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:39.222791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63293b6a-9796-4646-b2f6-070b4cd0c35e admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:39.223706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63293b6a-9796-4646-b2f6-070b4cd0c35e admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:39.224885 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63293b6a-9796-4646-b2f6-070b4cd0c35e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:39.227633 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63293b6a-9796-4646-b2f6-070b4cd0c35e admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:39.244886 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 855/1703] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:39.249012 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc209a-e9e9-4f42-b9d3-0b94bf528223 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:39.286526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140440f3-4748-47f8-a3bf-3a66eae42d7a None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:39.286826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140440f3-4748-47f8-a3bf-3a66eae42d7a None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:39.287130 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140440f3-4748-47f8-a3bf-3a66eae42d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:39.292879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc209a-e9e9-4f42-b9d3-0b94bf528223 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:39.311402 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5147ef-d0f0-4164-94b3-019536c68314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:39.312820 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5147ef-d0f0-4164-94b3-019536c68314 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:39.313101 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5147ef-d0f0-4164-94b3-019536c68314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:39.316711 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5147ef-d0f0-4164-94b3-019536c68314 admin admin] PATH_INFO: `/v3/projects/bfec858a072145f3a7b7ac593280e7e0/users/77f2e82d6d5e437baf931c4b5e78c551/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:39.377465 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5147ef-d0f0-4164-94b3-019536c68314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfec858a072145f3a7b7ac593280e7e0, user_id=77f2e82d6d5e437baf931c4b5e78c551, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:39.390066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-140440f3-4748-47f8-a3bf-3a66eae42d7a None None] MFA Rules not processed for user `d10b50451d4b48f0b2377626ed609de4`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:39.392614 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5147ef-d0f0-4164-94b3-019536c68314 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:39.395354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-140440f3-4748-47f8-a3bf-3a66eae42d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:39.461184 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 849/1704] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:20:39 2022] PUT /identity/v3/projects/bfec858a072145f3a7b7ac593280e7e0/users/77f2e82d6d5e437baf931c4b5e78c551/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:39.477005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c24e4-91f6-49d6-8896-c7c75cc83c2a None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:39.477388 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c24e4-91f6-49d6-8896-c7c75cc83c2a None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:39.477737 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c24e4-91f6-49d6-8896-c7c75cc83c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:39.549448 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-7e4c24e4-91f6-49d6-8896-c7c75cc83c2a None None] MFA Rules not processed for user `77f2e82d6d5e437baf931c4b5e78c551`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:39.550749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-7e4c24e4-91f6-49d6-8896-c7c75cc83c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:39.609333 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 856/1705] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:39.610248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140440f3-4748-47f8-a3bf-3a66eae42d7a None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:39.616724 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-140440f3-4748-47f8-a3bf-3a66eae42d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:39.659474 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140440f3-4748-47f8-a3bf-3a66eae42d7a None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:39.762926 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f9b1b-513a-4753-a0b8-9bad3424754a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:39.764043 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f9b1b-513a-4753-a0b8-9bad3424754a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:39.764389 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f9b1b-513a-4753-a0b8-9bad3424754a admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:39.764672 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f9b1b-513a-4753-a0b8-9bad3424754a admin admin] PATH_INFO: `/v3/projects/4b0b9ee8b63d4b7d9cfe7011aff85564` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:39.766073 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f9b1b-513a-4753-a0b8-9bad3424754a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b0b9ee8b63d4b7d9cfe7011aff85564)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:39.768548 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f9b1b-513a-4753-a0b8-9bad3424754a admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:39.780895 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 850/1706] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:20:39.784892 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c94b22-4bd2-432b-a496-102ed14c55d5 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:39.792606 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c94b22-4bd2-432b-a496-102ed14c55d5 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:39.793240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c94b22-4bd2-432b-a496-102ed14c55d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:39.814520 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:39.840540 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-c8c94b22-4bd2-432b-a496-102ed14c55d5 None None] MFA Rules not processed for user `d10b50451d4b48f0b2377626ed609de4`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:39.844366 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c8c94b22-4bd2-432b-a496-102ed14c55d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:39.872484 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5f8f9b1b-513a-4753-a0b8-9bad3424754a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b0b9ee8b63d4b7d9cfe7011aff85564', 'request_id': 'req-5f8f9b1b-513a-4753-a0b8-9bad3424754a'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:39.888151 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5f8f9b1b-513a-4753-a0b8-9bad3424754a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b0b9ee8b63d4b7d9cfe7011aff85564', 'request_id': 'req-5f8f9b1b-513a-4753-a0b8-9bad3424754a'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:39.900748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5f8f9b1b-513a-4753-a0b8-9bad3424754a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b0b9ee8b63d4b7d9cfe7011aff85564', 'request_id': 'req-5f8f9b1b-513a-4753-a0b8-9bad3424754a'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:39.921882 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5f8f9b1b-513a-4753-a0b8-9bad3424754a admin admin] The token cache is being invalidate because project 4b0b9ee8b63d4b7d9cfe7011aff85564 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:20:39.922272 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5f8f9b1b-513a-4753-a0b8-9bad3424754a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:39.923009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5f8f9b1b-513a-4753-a0b8-9bad3424754a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:39.940214 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 857/1707] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:20:39 2022] DELETE /identity/v3/projects/4b0b9ee8b63d4b7d9cfe7011aff85564 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:39.943778 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f9b1b-513a-4753-a0b8-9bad3424754a admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:39.948242 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-5f8f9b1b-513a-4753-a0b8-9bad3424754a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:40.004273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 851/1708] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:40.005680 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c94b22-4bd2-432b-a496-102ed14c55d5 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:40.011680 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c8c94b22-4bd2-432b-a496-102ed14c55d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:40.075710 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f9b1b-513a-4753-a0b8-9bad3424754a admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:40.089733 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329927ed-5391-48c3-804a-d115223a3d01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:40.090869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329927ed-5391-48c3-804a-d115223a3d01 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:40.092763 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329927ed-5391-48c3-804a-d115223a3d01 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:40.093195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329927ed-5391-48c3-804a-d115223a3d01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:40.099909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:40.103054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-329927ed-5391-48c3-804a-d115223a3d01 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:40.130305 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c94b22-4bd2-432b-a496-102ed14c55d5 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:40.137623 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917fd9c-eacf-4d71-8067-94306636cdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:40.139180 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917fd9c-eacf-4d71-8067-94306636cdab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:40.139665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917fd9c-eacf-4d71-8067-94306636cdab admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:40.140205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917fd9c-eacf-4d71-8067-94306636cdab admin admin] PATH_INFO: `/v3/users/bf0ecf872ce3405ab0b296c3568306cb` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:40.143087 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917fd9c-eacf-4d71-8067-94306636cdab admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf0ecf872ce3405ab0b296c3568306cb)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:40.146350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917fd9c-eacf-4d71-8067-94306636cdab admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:40.232479 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9917fd9c-eacf-4d71-8067-94306636cdab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf0ecf872ce3405ab0b296c3568306cb'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:40.253317 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9917fd9c-eacf-4d71-8067-94306636cdab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf0ecf872ce3405ab0b296c3568306cb'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:40.263004 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9917fd9c-eacf-4d71-8067-94306636cdab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf0ecf872ce3405ab0b296c3568306cb'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:40.275682 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9917fd9c-eacf-4d71-8067-94306636cdab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf0ecf872ce3405ab0b296c3568306cb'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:40.277120 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329927ed-5391-48c3-804a-d115223a3d01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:40.277977 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9917fd9c-eacf-4d71-8067-94306636cdab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf0ecf872ce3405ab0b296c3568306cb'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:40.281245 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329927ed-5391-48c3-804a-d115223a3d01 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:40.302524 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 852/1709] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:20:40 2022] DELETE /identity/v3/users/bf0ecf872ce3405ab0b296c3568306cb => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:40.303279 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917fd9c-eacf-4d71-8067-94306636cdab admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:40.305616 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-9917fd9c-eacf-4d71-8067-94306636cdab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:40.317270 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 858/1710] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:40.399085 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917fd9c-eacf-4d71-8067-94306636cdab admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:40.405729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6098e4c-ad7a-4dac-8f8f-1f3c904f6baa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:40.406866 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6098e4c-ad7a-4dac-8f8f-1f3c904f6baa service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:40.407894 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6098e4c-ad7a-4dac-8f8f-1f3c904f6baa service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:40.408402 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6098e4c-ad7a-4dac-8f8f-1f3c904f6baa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:40.410544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-d6098e4c-ad7a-4dac-8f8f-1f3c904f6baa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:40.558865 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6098e4c-ad7a-4dac-8f8f-1f3c904f6baa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:40.560526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6098e4c-ad7a-4dac-8f8f-1f3c904f6baa service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:40.565048 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:40.590079 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 853/1711] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:40.901519 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329927ed-5391-48c3-804a-d115223a3d01 service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:40.920692 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329927ed-5391-48c3-804a-d115223a3d01 service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:41.101648 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9df25-b70e-4f59-9917-3ff3772d4601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:41.102313 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9df25-b70e-4f59-9917-3ff3772d4601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:41.102575 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9df25-b70e-4f59-9917-3ff3772d4601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:41.102842 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9df25-b70e-4f59-9917-3ff3772d4601 admin admin] PATH_INFO: `/v3/projects/21a5a52e9fa043759ce05af5ef7f9fd8` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:41.106822 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9df25-b70e-4f59-9917-3ff3772d4601 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21a5a52e9fa043759ce05af5ef7f9fd8)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:41.108866 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9df25-b70e-4f59-9917-3ff3772d4601 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:41.130291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:41.301392 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-29d9df25-b70e-4f59-9917-3ff3772d4601 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21a5a52e9fa043759ce05af5ef7f9fd8', 'request_id': 'req-29d9df25-b70e-4f59-9917-3ff3772d4601'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:41.315177 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-29d9df25-b70e-4f59-9917-3ff3772d4601 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21a5a52e9fa043759ce05af5ef7f9fd8', 'request_id': 'req-29d9df25-b70e-4f59-9917-3ff3772d4601'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:41.331614 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-29d9df25-b70e-4f59-9917-3ff3772d4601 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21a5a52e9fa043759ce05af5ef7f9fd8', 'request_id': 'req-29d9df25-b70e-4f59-9917-3ff3772d4601'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:41.361065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-29d9df25-b70e-4f59-9917-3ff3772d4601 admin admin] The token cache is being invalidate because project 21a5a52e9fa043759ce05af5ef7f9fd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:20:41.361065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-29d9df25-b70e-4f59-9917-3ff3772d4601 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:41.361759 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-29d9df25-b70e-4f59-9917-3ff3772d4601 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:41.363667 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 859/1712] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:20:40 2022] DELETE /identity/v3/projects/21a5a52e9fa043759ce05af5ef7f9fd8 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:41.369626 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5df02c-1881-4dbc-94da-1684c6f06ba2 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:41.371533 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5df02c-1881-4dbc-94da-1684c6f06ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:41.372040 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5df02c-1881-4dbc-94da-1684c6f06ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:41.415612 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-4b5df02c-1881-4dbc-94da-1684c6f06ba2 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:41.418790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-4b5df02c-1881-4dbc-94da-1684c6f06ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:41.615874 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6badec-4191-4ce3-ad60-86aa17a46281 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:41.618179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6badec-4191-4ce3-ad60-86aa17a46281 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:41.619261 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6badec-4191-4ce3-ad60-86aa17a46281 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:41.621434 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 860/1713] 10.222.0.67 () {54 vars in 912 bytes} [Wed Aug 31 00:20:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:20:41.628576 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa286cf7-ab95-45bf-8c1b-9b351cc850dc None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:41.628864 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa286cf7-ab95-45bf-8c1b-9b351cc850dc None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:41.629142 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa286cf7-ab95-45bf-8c1b-9b351cc850dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:41.704330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 854/1714] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:41.713601 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009e608-ff22-48ac-a785-96ac87f236e3 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:41.713811 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009e608-ff22-48ac-a785-96ac87f236e3 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:41.714016 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009e608-ff22-48ac-a785-96ac87f236e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:41.774957 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-aa286cf7-ab95-45bf-8c1b-9b351cc850dc None None] MFA Rules not processed for user `05c0d9554cf94c41a2cd709cab378e53`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:41.776230 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-aa286cf7-ab95-45bf-8c1b-9b351cc850dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:41.783850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-b009e608-ff22-48ac-a785-96ac87f236e3 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:41.785083 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b009e608-ff22-48ac-a785-96ac87f236e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:41.813635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 855/1715] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:20:41.824120 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009e608-ff22-48ac-a785-96ac87f236e3 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:41.844411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009e608-ff22-48ac-a785-96ac87f236e3 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:41.863370 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9490d91-0e04-4a91-b1eb-1583421866e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:41.864002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9490d91-0e04-4a91-b1eb-1583421866e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:41.864281 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9490d91-0e04-4a91-b1eb-1583421866e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:41.864565 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9490d91-0e04-4a91-b1eb-1583421866e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:41.865400 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9490d91-0e04-4a91-b1eb-1583421866e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:20:41.865703 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9490d91-0e04-4a91-b1eb-1583421866e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:41.868237 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9490d91-0e04-4a91-b1eb-1583421866e7 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:41.930196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 856/1716] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:20:41.935371 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9490d91-0e04-4a91-b1eb-1583421866e7 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:41.956061 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 861/1717] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 31 00:20:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:20:41.963563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9490d91-0e04-4a91-b1eb-1583421866e7 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:41.982851 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa286cf7-ab95-45bf-8c1b-9b351cc850dc None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:41.984419 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-aa286cf7-ab95-45bf-8c1b-9b351cc850dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:42.004123 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6ad1a-1992-472c-a883-f12ae899ff28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:42.004699 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6ad1a-1992-472c-a883-f12ae899ff28 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:42.004982 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6ad1a-1992-472c-a883-f12ae899ff28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:42.005251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6ad1a-1992-472c-a883-f12ae899ff28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:42.005890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a6ad1a-1992-472c-a883-f12ae899ff28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:42.008541 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a6ad1a-1992-472c-a883-f12ae899ff28 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:42.023238 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa286cf7-ab95-45bf-8c1b-9b351cc850dc None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:42.038093 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2c656-fd4e-4d88-9670-75cb8584a49f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:42.039099 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2c656-fd4e-4d88-9670-75cb8584a49f service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:42.039403 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2c656-fd4e-4d88-9670-75cb8584a49f service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:42.039674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2c656-fd4e-4d88-9670-75cb8584a49f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:42.072555 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab2c656-fd4e-4d88-9670-75cb8584a49f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:42.074035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab2c656-fd4e-4d88-9670-75cb8584a49f service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:42.100139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 862/1718] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:42.105348 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 857/1719] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:20:41 2022] POST /identity/v3/projects => generated 365 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:42.109055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2c656-fd4e-4d88-9670-75cb8584a49f service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:42.130161 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2c656-fd4e-4d88-9670-75cb8584a49f service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:42.242380 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36747b5c-a2c6-4368-924f-f5c275de58eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:42.243517 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36747b5c-a2c6-4368-924f-f5c275de58eb admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:42.244009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36747b5c-a2c6-4368-924f-f5c275de58eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:42.244456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36747b5c-a2c6-4368-924f-f5c275de58eb admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:42.245322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36747b5c-a2c6-4368-924f-f5c275de58eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:42.249637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36747b5c-a2c6-4368-924f-f5c275de58eb admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:42.288900 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 863/1720] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:20:42 2022] POST /identity/v3/users => generated 283 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:42.293256 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6ad1a-1992-472c-a883-f12ae899ff28 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:42.310192 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6ad1a-1992-472c-a883-f12ae899ff28 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:42.327372 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc416af-2e58-4d61-937b-d3d8991d1471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:42.340350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc416af-2e58-4d61-937b-d3d8991d1471 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:42.341541 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc416af-2e58-4d61-937b-d3d8991d1471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:42.342393 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc416af-2e58-4d61-937b-d3d8991d1471 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:42.345047 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc416af-2e58-4d61-937b-d3d8991d1471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:42.349510 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc416af-2e58-4d61-937b-d3d8991d1471 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:42.380381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 858/1721] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:42.385821 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36747b5c-a2c6-4368-924f-f5c275de58eb admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:42.402339 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36747b5c-a2c6-4368-924f-f5c275de58eb admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:42.443727 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3550a695-9ef6-448b-8ae3-e6caf78a1a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:42.453088 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550a695-9ef6-448b-8ae3-e6caf78a1a6e admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:42.455440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550a695-9ef6-448b-8ae3-e6caf78a1a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:42.455736 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550a695-9ef6-448b-8ae3-e6caf78a1a6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:42.462035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3550a695-9ef6-448b-8ae3-e6caf78a1a6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:42.467664 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3550a695-9ef6-448b-8ae3-e6caf78a1a6e admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:42.478330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:42.506080 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc416af-2e58-4d61-937b-d3d8991d1471 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:42.522963 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc416af-2e58-4d61-937b-d3d8991d1471 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:42.525856 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 864/1722] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:42.547298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88efe2b0-1029-4fa1-a6bf-5d8715acfa6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:42.547977 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88efe2b0-1029-4fa1-a6bf-5d8715acfa6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:42.548410 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88efe2b0-1029-4fa1-a6bf-5d8715acfa6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:42.548833 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88efe2b0-1029-4fa1-a6bf-5d8715acfa6c admin admin] PATH_INFO: `/v3/projects/891c3e6e45d945e59f6b41adfb62b381/users/4fdcfd233bfd4d6ca1bc16f14f069469/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:42.596880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88efe2b0-1029-4fa1-a6bf-5d8715acfa6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=891c3e6e45d945e59f6b41adfb62b381, user_id=4fdcfd233bfd4d6ca1bc16f14f069469, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:42.600065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88efe2b0-1029-4fa1-a6bf-5d8715acfa6c admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:42.652128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 859/1723] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:20:42 2022] PUT /identity/v3/projects/891c3e6e45d945e59f6b41adfb62b381/users/4fdcfd233bfd4d6ca1bc16f14f069469/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:42.653159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9342e8e-fa5c-4723-ab1f-a105ca763829 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:42.653457 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9342e8e-fa5c-4723-ab1f-a105ca763829 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:42.653740 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9342e8e-fa5c-4723-ab1f-a105ca763829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:42.768281 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-c9342e8e-fa5c-4723-ab1f-a105ca763829 None None] MFA Rules not processed for user `4fdcfd233bfd4d6ca1bc16f14f069469`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:42.776613 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-c9342e8e-fa5c-4723-ab1f-a105ca763829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:42.875796 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1642c-5985-49d2-905f-df2ad76761fc None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:42.876632 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1642c-5985-49d2-905f-df2ad76761fc None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:42.877088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1642c-5985-49d2-905f-df2ad76761fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:42.946679 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-9dd1642c-5985-49d2-905f-df2ad76761fc None None] MFA Rules not processed for user `6d1a15d59d5d476ca6e0ab5176d310cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:42.948932 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-9dd1642c-5985-49d2-905f-df2ad76761fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:43.092806 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 865/1724] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 441 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:43.098693 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9342e8e-fa5c-4723-ab1f-a105ca763829 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:43.119906 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 860/1725] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:20:43.130668 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1642c-5985-49d2-905f-df2ad76761fc None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:43.133321 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-9dd1642c-5985-49d2-905f-df2ad76761fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:43.157523 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:43.318731 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1642c-5985-49d2-905f-df2ad76761fc None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:43.335912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c50e73-c305-4b26-84e9-03efa5442f4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:43.336423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c50e73-c305-4b26-84e9-03efa5442f4a service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:43.336708 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c50e73-c305-4b26-84e9-03efa5442f4a service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:43.336984 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c50e73-c305-4b26-84e9-03efa5442f4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:43.347268 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c50e73-c305-4b26-84e9-03efa5442f4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:43.348859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c50e73-c305-4b26-84e9-03efa5442f4a service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:43.351510 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9342e8e-fa5c-4723-ab1f-a105ca763829 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:43.384167 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc35a85-3fa5-4ea6-8c2b-9a3aa46e7e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:43.384506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 861/1726] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:20:43.387214 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc35a85-3fa5-4ea6-8c2b-9a3aa46e7e30 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:43.387601 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc35a85-3fa5-4ea6-8c2b-9a3aa46e7e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:43.391368 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc35a85-3fa5-4ea6-8c2b-9a3aa46e7e30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:43.392157 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc35a85-3fa5-4ea6-8c2b-9a3aa46e7e30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:43.394180 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc35a85-3fa5-4ea6-8c2b-9a3aa46e7e30 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:43.459686 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 866/1727] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:20:43 2022] POST /identity/v3/projects => generated 365 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:43.463200 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c50e73-c305-4b26-84e9-03efa5442f4a service nova] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:43.486826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc35a85-3fa5-4ea6-8c2b-9a3aa46e7e30 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:43.488889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-0cc35a85-3fa5-4ea6-8c2b-9a3aa46e7e30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:43.497645 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c50e73-c305-4b26-84e9-03efa5442f4a service nova] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:43.617565 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2555a76-0a92-4cbe-b7cf-10fa0abc55a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:43.618202 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2555a76-0a92-4cbe-b7cf-10fa0abc55a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:43.618480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2555a76-0a92-4cbe-b7cf-10fa0abc55a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:43.618927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2555a76-0a92-4cbe-b7cf-10fa0abc55a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:43.619795 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2555a76-0a92-4cbe-b7cf-10fa0abc55a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:43.622572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2555a76-0a92-4cbe-b7cf-10fa0abc55a5 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:43.652837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 862/1728] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:20:43 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:43.657248 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2555a76-0a92-4cbe-b7cf-10fa0abc55a5 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:43.671007 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc35a85-3fa5-4ea6-8c2b-9a3aa46e7e30 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:43.681218 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4adf588-33ae-478c-b13c-c6266eee59de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdacdcb2451742938164fdd44cdbaebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:43.681426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4adf588-33ae-478c-b13c-c6266eee59de service placement] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:43.681979 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4adf588-33ae-478c-b13c-c6266eee59de service placement] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:43.682190 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4adf588-33ae-478c-b13c-c6266eee59de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:43.684606 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-c4adf588-33ae-478c-b13c-c6266eee59de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:43.697454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2555a76-0a92-4cbe-b7cf-10fa0abc55a5 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:43.708760 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4adf588-33ae-478c-b13c-c6266eee59de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:43.711312 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4adf588-33ae-478c-b13c-c6266eee59de service placement] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:43.715512 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f02bb6-0b96-4ee3-8e20-f505d6861908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:43.724193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f02bb6-0b96-4ee3-8e20-f505d6861908 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:43.724498 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f02bb6-0b96-4ee3-8e20-f505d6861908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:43.724784 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f02bb6-0b96-4ee3-8e20-f505d6861908 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:43.726955 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f02bb6-0b96-4ee3-8e20-f505d6861908 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:43.734064 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f02bb6-0b96-4ee3-8e20-f505d6861908 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:43.738265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 867/1729] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:43.780432 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 863/1730] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:43.785579 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4adf588-33ae-478c-b13c-c6266eee59de service placement] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:43.793255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:43.802530 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4adf588-33ae-478c-b13c-c6266eee59de service placement] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:43.823914 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e0669-7f81-4935-a20d-f4084ea377b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:43.826753 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e0669-7f81-4935-a20d-f4084ea377b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:43.827070 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e0669-7f81-4935-a20d-f4084ea377b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:43.831262 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e0669-7f81-4935-a20d-f4084ea377b3 admin admin] PATH_INFO: `/v3/projects/c8d19f9281f54813b356c9e10ee6924b/users/7e0eba84e85f4acfa9c828c8aa76538d/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:43.870188 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e0669-7f81-4935-a20d-f4084ea377b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8d19f9281f54813b356c9e10ee6924b, user_id=7e0eba84e85f4acfa9c828c8aa76538d, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:43.874021 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e0669-7f81-4935-a20d-f4084ea377b3 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:43.908335 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 868/1731] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:20:43 2022] PUT /identity/v3/projects/c8d19f9281f54813b356c9e10ee6924b/users/7e0eba84e85f4acfa9c828c8aa76538d/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:43.912488 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f02bb6-0b96-4ee3-8e20-f505d6861908 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:43.928506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f02bb6-0b96-4ee3-8e20-f505d6861908 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:44.053068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be0072-08ef-4a2e-99df-32b2ab0306c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:44.055768 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be0072-08ef-4a2e-99df-32b2ab0306c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:44.056020 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be0072-08ef-4a2e-99df-32b2ab0306c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:44.056302 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be0072-08ef-4a2e-99df-32b2ab0306c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:44.059367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79be0072-08ef-4a2e-99df-32b2ab0306c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:44.061561 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79be0072-08ef-4a2e-99df-32b2ab0306c9 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:44.080372 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 864/1732] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:44.082109 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e0669-7f81-4935-a20d-f4084ea377b3 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:44.097208 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e0669-7f81-4935-a20d-f4084ea377b3 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:44.108867 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c0f76-f67f-40b7-a616-6f1d903e6c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:44.108867 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c0f76-f67f-40b7-a616-6f1d903e6c1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:44.109089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c0f76-f67f-40b7-a616-6f1d903e6c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:44.109159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c0f76-f67f-40b7-a616-6f1d903e6c1e admin admin] PATH_INFO: `/v3/projects/c8d19f9281f54813b356c9e10ee6924b/users/7e0eba84e85f4acfa9c828c8aa76538d/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:44.115536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c0f76-f67f-40b7-a616-6f1d903e6c1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8d19f9281f54813b356c9e10ee6924b, user_id=7e0eba84e85f4acfa9c828c8aa76538d, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:44.119738 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c0f76-f67f-40b7-a616-6f1d903e6c1e admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:44.151739 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 869/1733] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:20:44 2022] PUT /identity/v3/projects/c8d19f9281f54813b356c9e10ee6924b/users/7e0eba84e85f4acfa9c828c8aa76538d/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:20:44.161338 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d960d9-c251-4047-98f3-f5d286b633e1 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:44.161687 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d960d9-c251-4047-98f3-f5d286b633e1 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:44.161973 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d960d9-c251-4047-98f3-f5d286b633e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:44.204167 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:44.221145 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-f6d960d9-c251-4047-98f3-f5d286b633e1 None None] MFA Rules not processed for user `7e0eba84e85f4acfa9c828c8aa76538d`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:44.222958 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f6d960d9-c251-4047-98f3-f5d286b633e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:44.407077 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 865/1734] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:44 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:44.420070 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c0f76-f67f-40b7-a616-6f1d903e6c1e admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:44.422308 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-172c0f76-f67f-40b7-a616-6f1d903e6c1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:44.668660 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c0f76-f67f-40b7-a616-6f1d903e6c1e admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:44.696781 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030694ca-3d75-4f03-8d6c-d5a51ea9ced2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:44.697445 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030694ca-3d75-4f03-8d6c-d5a51ea9ced2 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:44.697711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030694ca-3d75-4f03-8d6c-d5a51ea9ced2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:44.697985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030694ca-3d75-4f03-8d6c-d5a51ea9ced2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:44.740237 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030694ca-3d75-4f03-8d6c-d5a51ea9ced2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:44.747028 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030694ca-3d75-4f03-8d6c-d5a51ea9ced2 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:44.789813 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 870/1735] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:44 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:44.941749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d960d9-c251-4047-98f3-f5d286b633e1 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:44.957912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d960d9-c251-4047-98f3-f5d286b633e1 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:44.972720 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99fc38-48bb-493f-96a9-e3b3606268e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:44.972720 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99fc38-48bb-493f-96a9-e3b3606268e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:44.972720 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99fc38-48bb-493f-96a9-e3b3606268e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:44.972720 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99fc38-48bb-493f-96a9-e3b3606268e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:45.216014 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a99fc38-48bb-493f-96a9-e3b3606268e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:45.220729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a99fc38-48bb-493f-96a9-e3b3606268e2 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:45.299496 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 866/1736] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:47.251564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad6717-c5f0-4e86-857e-b34d80f8882b None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:47.252130 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad6717-c5f0-4e86-857e-b34d80f8882b None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:47.252360 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad6717-c5f0-4e86-857e-b34d80f8882b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:47.282846 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-5aad6717-c5f0-4e86-857e-b34d80f8882b None None] MFA Rules not processed for user `6d1a15d59d5d476ca6e0ab5176d310cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:47.283866 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-5aad6717-c5f0-4e86-857e-b34d80f8882b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:47.378223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11860fae-e4b9-4123-8c3e-2e7a3d70431d None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:47.379165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11860fae-e4b9-4123-8c3e-2e7a3d70431d None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:47.379646 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11860fae-e4b9-4123-8c3e-2e7a3d70431d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:47.399796 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 871/1737] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:47.427856 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-11860fae-e4b9-4123-8c3e-2e7a3d70431d None None] MFA Rules not processed for user `7d7582f9cdd646f39404f7adc99ad9ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:47.430356 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-11860fae-e4b9-4123-8c3e-2e7a3d70431d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:47.505382 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1007de1-adf4-4745-a2eb-45ac972a55f3 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:47.505936 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1007de1-adf4-4745-a2eb-45ac972a55f3 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:47.506441 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1007de1-adf4-4745-a2eb-45ac972a55f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:47.553377 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-b1007de1-adf4-4745-a2eb-45ac972a55f3 None None] MFA Rules not processed for user `3ff39cd0e766400f9d25e3ad465c4faf`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:47.555678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b1007de1-adf4-4745-a2eb-45ac972a55f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:47.591511 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 867/1738] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:47 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:20:47.607977 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11860fae-e4b9-4123-8c3e-2e7a3d70431d None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:47.610258 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-11860fae-e4b9-4123-8c3e-2e7a3d70431d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:47.662762 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:47.680483 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 872/1739] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:47 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:47.689641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e8515-55da-407b-964a-c9595d7986b6 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:47.690117 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e8515-55da-407b-964a-c9595d7986b6 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:47.690746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e8515-55da-407b-964a-c9595d7986b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:47.699001 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:47.716297 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-511e8515-55da-407b-964a-c9595d7986b6 None None] MFA Rules not processed for user `3ff39cd0e766400f9d25e3ad465c4faf`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:47.717910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-511e8515-55da-407b-964a-c9595d7986b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:47.728400 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 873/1740] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:47 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:47.754471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11860fae-e4b9-4123-8c3e-2e7a3d70431d None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:47.760967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079be73-3e07-46eb-9861-a03971a32561 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:47.761869 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079be73-3e07-46eb-9861-a03971a32561 service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:47.762371 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079be73-3e07-46eb-9861-a03971a32561 service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:47.762746 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079be73-3e07-46eb-9861-a03971a32561 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:47.773679 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0079be73-3e07-46eb-9861-a03971a32561 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:47.775667 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0079be73-3e07-46eb-9861-a03971a32561 service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:47.787085 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 868/1741] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:47 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:47.916649 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e8515-55da-407b-964a-c9595d7986b6 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:47.949737 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e8515-55da-407b-964a-c9595d7986b6 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:47.965160 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398bc54-bb93-4c83-83e9-5601a3781611 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:47.965817 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398bc54-bb93-4c83-83e9-5601a3781611 service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:47.966100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398bc54-bb93-4c83-83e9-5601a3781611 service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:47.976445 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398bc54-bb93-4c83-83e9-5601a3781611 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:47.997627 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96018d1b-4621-4ba7-bda8-6e3a856bfa3a None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:47.997627 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96018d1b-4621-4ba7-bda8-6e3a856bfa3a None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:47.997850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96018d1b-4621-4ba7-bda8-6e3a856bfa3a None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:47.999667 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 869/1742] 10.222.0.67 () {54 vars in 912 bytes} [Wed Aug 31 00:20:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:20:48.008755 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398bc54-bb93-4c83-83e9-5601a3781611 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:48.011326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398bc54-bb93-4c83-83e9-5601a3781611 service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:48.034459 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda79d4-0912-41c3-8096-df95a47216f4 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:48.034746 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda79d4-0912-41c3-8096-df95a47216f4 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:48.035027 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda79d4-0912-41c3-8096-df95a47216f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:48.036375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 874/1743] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:47 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:20:48.121239 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-1bda79d4-0912-41c3-8096-df95a47216f4 None None] MFA Rules not processed for user `05c0d9554cf94c41a2cd709cab378e53`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:48.129678 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-1bda79d4-0912-41c3-8096-df95a47216f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:48.307143 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 870/1744] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 31 00:20:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:48.480467 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398bc54-bb93-4c83-83e9-5601a3781611 service nova] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:48.498594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398bc54-bb93-4c83-83e9-5601a3781611 service nova] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:48.503257 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05174a-464c-49bf-835b-d5ef6b68a729 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:48.504015 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05174a-464c-49bf-835b-d5ef6b68a729 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:48.504499 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05174a-464c-49bf-835b-d5ef6b68a729 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:48.505745 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05174a-464c-49bf-835b-d5ef6b68a729 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:48.526627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d05174a-464c-49bf-835b-d5ef6b68a729 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:48.528945 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d05174a-464c-49bf-835b-d5ef6b68a729 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:48.550055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 875/1745] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:48.628514 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc760b3-0104-44a3-9902-064ebeebfece None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:48.629694 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc760b3-0104-44a3-9902-064ebeebfece None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:48.630105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc760b3-0104-44a3-9902-064ebeebfece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:48.747325 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-afc760b3-0104-44a3-9902-064ebeebfece None None] MFA Rules not processed for user `a8fb7a9acefd4b418c992661037c2ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:48.752246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-afc760b3-0104-44a3-9902-064ebeebfece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:48.963839 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 871/1746] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 31 00:20:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:49.265819 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05174a-464c-49bf-835b-d5ef6b68a729 service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:49.267901 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-8d05174a-464c-49bf-835b-d5ef6b68a729 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:49.632285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05174a-464c-49bf-835b-d5ef6b68a729 service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:49.645336 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4117497-3f03-497e-b839-321a6587c5e5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd3cfbc8ebd4c5a83bef09efe10bc91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:49.647000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4117497-3f03-497e-b839-321a6587c5e5 service swift] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:49.648007 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4117497-3f03-497e-b839-321a6587c5e5 service swift] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:49.648490 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4117497-3f03-497e-b839-321a6587c5e5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:49.663913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4117497-3f03-497e-b839-321a6587c5e5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:49.665780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4117497-3f03-497e-b839-321a6587c5e5 service swift] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:49.676688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 876/1747] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 31 00:20:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:20:52.140001 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc760b3-0104-44a3-9902-064ebeebfece None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:52.142521 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-afc760b3-0104-44a3-9902-064ebeebfece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:52.290432 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:52.484377 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc760b3-0104-44a3-9902-064ebeebfece None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:52.502059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99851be-cd9b-49eb-bda8-0b4ffaf783cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:52.504187 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99851be-cd9b-49eb-bda8-0b4ffaf783cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:52.505320 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99851be-cd9b-49eb-bda8-0b4ffaf783cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:52.506140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99851be-cd9b-49eb-bda8-0b4ffaf783cb admin admin] PATH_INFO: `/v3/users/b8a88388dd8a45249e975fc92803778e` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:52.512357 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99851be-cd9b-49eb-bda8-0b4ffaf783cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8a88388dd8a45249e975fc92803778e)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:52.549547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99851be-cd9b-49eb-bda8-0b4ffaf783cb admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:52.711524 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcc7c3-c3c4-42f7-aff4-32ee63596b77 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:52.718738 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcc7c3-c3c4-42f7-aff4-32ee63596b77 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:52.720354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcc7c3-c3c4-42f7-aff4-32ee63596b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:52.751347 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b99851be-cd9b-49eb-bda8-0b4ffaf783cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8a88388dd8a45249e975fc92803778e'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:52.774581 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b99851be-cd9b-49eb-bda8-0b4ffaf783cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8a88388dd8a45249e975fc92803778e'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:52.793014 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b99851be-cd9b-49eb-bda8-0b4ffaf783cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8a88388dd8a45249e975fc92803778e'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:52.811217 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b99851be-cd9b-49eb-bda8-0b4ffaf783cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8a88388dd8a45249e975fc92803778e'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:52.825263 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b99851be-cd9b-49eb-bda8-0b4ffaf783cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8a88388dd8a45249e975fc92803778e'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:52.846616 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 872/1748] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:20:52 2022] DELETE /identity/v3/users/b8a88388dd8a45249e975fc92803778e => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:52.850978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ea866-c0d6-430f-9284-24420dbbf8a4 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:52.853102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ea866-c0d6-430f-9284-24420dbbf8a4 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:52.853102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ea866-c0d6-430f-9284-24420dbbf8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:52.867912 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-87bcc7c3-c3c4-42f7-aff4-32ee63596b77 None None] MFA Rules not processed for user `05c0d9554cf94c41a2cd709cab378e53`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:52.878097 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-87bcc7c3-c3c4-42f7-aff4-32ee63596b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:52.975400 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-f31ea866-c0d6-430f-9284-24420dbbf8a4 None None] MFA Rules not processed for user `3ff39cd0e766400f9d25e3ad465c4faf`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:52.977053 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f31ea866-c0d6-430f-9284-24420dbbf8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:53.123906 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 877/1749] 10.222.0.58 () {58 vars in 1015 bytes} [Wed Aug 31 00:20:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:53.127848 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd23ec-16f7-49a5-85da-1dd4fa0917e5 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:53.129017 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd23ec-16f7-49a5-85da-1dd4fa0917e5 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:53.129464 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd23ec-16f7-49a5-85da-1dd4fa0917e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:53.188934 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-dbcd23ec-16f7-49a5-85da-1dd4fa0917e5 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:53.191528 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-dbcd23ec-16f7-49a5-85da-1dd4fa0917e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:53.252914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 873/1750] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:53.253515 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be402a0-b4b7-43db-8634-a6bef793c3a4 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:53.254663 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be402a0-b4b7-43db-8634-a6bef793c3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:53.255665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be402a0-b4b7-43db-8634-a6bef793c3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:53.316786 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-9be402a0-b4b7-43db-8634-a6bef793c3a4 None None] MFA Rules not processed for user `7d7582f9cdd646f39404f7adc99ad9ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:53.321764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-9be402a0-b4b7-43db-8634-a6bef793c3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:53.345310 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:53.401012 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 878/1751] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:53.418548 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd23ec-16f7-49a5-85da-1dd4fa0917e5 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:53.420351 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-dbcd23ec-16f7-49a5-85da-1dd4fa0917e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:53.450829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd23ec-16f7-49a5-85da-1dd4fa0917e5 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:53.462478 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4bf28c-da51-4956-b4f7-10b1e6cd7a70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:53.463731 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bf28c-da51-4956-b4f7-10b1e6cd7a70 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:53.464235 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bf28c-da51-4956-b4f7-10b1e6cd7a70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:53.464719 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bf28c-da51-4956-b4f7-10b1e6cd7a70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:53.499600 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:53.514128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 874/1752] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:53 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:53.519242 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4bf28c-da51-4956-b4f7-10b1e6cd7a70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:53.520583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4bf28c-da51-4956-b4f7-10b1e6cd7a70 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:53.571076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 879/1753] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:54.325002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be402a0-b4b7-43db-8634-a6bef793c3a4 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:54.326573 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-9be402a0-b4b7-43db-8634-a6bef793c3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:54.353335 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be402a0-b4b7-43db-8634-a6bef793c3a4 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:54.368455 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fbc92-0b6a-4aa0-9280-e244fe758f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:54.372784 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fbc92-0b6a-4aa0-9280-e244fe758f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:54.373147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fbc92-0b6a-4aa0-9280-e244fe758f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:54.373500 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fbc92-0b6a-4aa0-9280-e244fe758f78 admin admin] PATH_INFO: `/v3/projects/78407f2ba91a45b2b1573e877aeadbb1` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:54.382399 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fbc92-0b6a-4aa0-9280-e244fe758f78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78407f2ba91a45b2b1573e877aeadbb1)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:54.385543 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fbc92-0b6a-4aa0-9280-e244fe758f78 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:54.466067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e79fbc92-0b6a-4aa0-9280-e244fe758f78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78407f2ba91a45b2b1573e877aeadbb1', 'request_id': 'req-e79fbc92-0b6a-4aa0-9280-e244fe758f78'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:54.479061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e79fbc92-0b6a-4aa0-9280-e244fe758f78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78407f2ba91a45b2b1573e877aeadbb1', 'request_id': 'req-e79fbc92-0b6a-4aa0-9280-e244fe758f78'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:54.492271 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e79fbc92-0b6a-4aa0-9280-e244fe758f78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78407f2ba91a45b2b1573e877aeadbb1', 'request_id': 'req-e79fbc92-0b6a-4aa0-9280-e244fe758f78'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:54.519674 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e79fbc92-0b6a-4aa0-9280-e244fe758f78 admin admin] The token cache is being invalidate because project 78407f2ba91a45b2b1573e877aeadbb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:20:54.519971 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e79fbc92-0b6a-4aa0-9280-e244fe758f78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:54.520838 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e79fbc92-0b6a-4aa0-9280-e244fe758f78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:54.526276 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 875/1754] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:20:54 2022] DELETE /identity/v3/projects/78407f2ba91a45b2b1573e877aeadbb1 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:54.527840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4bf28c-da51-4956-b4f7-10b1e6cd7a70 service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:54.550217 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4bf28c-da51-4956-b4f7-10b1e6cd7a70 service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:54.658205 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b977b-9859-4500-b129-6b3bea60dd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:54.660316 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b977b-9859-4500-b129-6b3bea60dd46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:54.660763 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b977b-9859-4500-b129-6b3bea60dd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:54.662120 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b977b-9859-4500-b129-6b3bea60dd46 admin admin] PATH_INFO: `/v3/users/fa672d3d8cf544fe814e391f05c1e221` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:54.664493 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514b977b-9859-4500-b129-6b3bea60dd46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa672d3d8cf544fe814e391f05c1e221)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:54.667319 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514b977b-9859-4500-b129-6b3bea60dd46 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:54.749405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-514b977b-9859-4500-b129-6b3bea60dd46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa672d3d8cf544fe814e391f05c1e221'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:54.762067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-514b977b-9859-4500-b129-6b3bea60dd46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa672d3d8cf544fe814e391f05c1e221'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:54.771138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-514b977b-9859-4500-b129-6b3bea60dd46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa672d3d8cf544fe814e391f05c1e221'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:54.772386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-514b977b-9859-4500-b129-6b3bea60dd46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa672d3d8cf544fe814e391f05c1e221'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:54.773921 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-514b977b-9859-4500-b129-6b3bea60dd46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa672d3d8cf544fe814e391f05c1e221'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:54.801107 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 880/1755] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:20:54 2022] DELETE /identity/v3/users/fa672d3d8cf544fe814e391f05c1e221 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:20:55.333077 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fbc92-0b6a-4aa0-9280-e244fe758f78 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:55.334725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-e79fbc92-0b6a-4aa0-9280-e244fe758f78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:55.352854 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fbc92-0b6a-4aa0-9280-e244fe758f78 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:55.541118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf43bd9-28c3-43e9-9468-0035c4a72672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:55.542232 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf43bd9-28c3-43e9-9468-0035c4a72672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:55.542731 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf43bd9-28c3-43e9-9468-0035c4a72672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:55.543636 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf43bd9-28c3-43e9-9468-0035c4a72672 admin admin] PATH_INFO: `/v3/projects/044bb7d6941242828dd02edcb2f793c9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:55.545625 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf43bd9-28c3-43e9-9468-0035c4a72672 admin admin] RBAC: Authorizing `identity:delete_project(project_id=044bb7d6941242828dd02edcb2f793c9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:55.549920 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf43bd9-28c3-43e9-9468-0035c4a72672 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:55.644046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-acf43bd9-28c3-43e9-9468-0035c4a72672 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '044bb7d6941242828dd02edcb2f793c9', 'request_id': 'req-acf43bd9-28c3-43e9-9468-0035c4a72672'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:55.661486 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-acf43bd9-28c3-43e9-9468-0035c4a72672 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '044bb7d6941242828dd02edcb2f793c9', 'request_id': 'req-acf43bd9-28c3-43e9-9468-0035c4a72672'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:55.684478 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-acf43bd9-28c3-43e9-9468-0035c4a72672 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '044bb7d6941242828dd02edcb2f793c9', 'request_id': 'req-acf43bd9-28c3-43e9-9468-0035c4a72672'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:55.704443 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-acf43bd9-28c3-43e9-9468-0035c4a72672 admin admin] The token cache is being invalidate because project 044bb7d6941242828dd02edcb2f793c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:20:55.705245 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-acf43bd9-28c3-43e9-9468-0035c4a72672 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:55.706577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-acf43bd9-28c3-43e9-9468-0035c4a72672 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:55.709462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 876/1756] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:20:55 2022] DELETE /identity/v3/projects/044bb7d6941242828dd02edcb2f793c9 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:55.714887 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161d103-3184-4d1f-8f95-5b78d6ff37bd None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:55.715729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161d103-3184-4d1f-8f95-5b78d6ff37bd None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:55.715729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161d103-3184-4d1f-8f95-5b78d6ff37bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:55.751847 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-0161d103-3184-4d1f-8f95-5b78d6ff37bd None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:55.753360 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-0161d103-3184-4d1f-8f95-5b78d6ff37bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:55.847525 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:55.899382 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 881/1757] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:55.911996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79712be-5031-4f2a-8094-54287dc707a1 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:55.912526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79712be-5031-4f2a-8094-54287dc707a1 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:55.914135 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79712be-5031-4f2a-8094-54287dc707a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:55.950654 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-d79712be-5031-4f2a-8094-54287dc707a1 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:55.951892 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-d79712be-5031-4f2a-8094-54287dc707a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:55.963342 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 877/1758] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:55.967654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161d103-3184-4d1f-8f95-5b78d6ff37bd None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:55.987985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161d103-3184-4d1f-8f95-5b78d6ff37bd None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:56.008080 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216e97b-be4d-48c7-b761-a45674d27143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:56.009599 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216e97b-be4d-48c7-b761-a45674d27143 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:56.009599 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216e97b-be4d-48c7-b761-a45674d27143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:56.009599 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216e97b-be4d-48c7-b761-a45674d27143 admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:56.011023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5216e97b-be4d-48c7-b761-a45674d27143 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:20:56.011357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5216e97b-be4d-48c7-b761-a45674d27143 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:56.020388 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5216e97b-be4d-48c7-b761-a45674d27143 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:56.054669 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 882/1759] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:20:56.059767 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79712be-5031-4f2a-8094-54287dc707a1 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:56.072892 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79712be-5031-4f2a-8094-54287dc707a1 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:56.088995 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c7039-c1e1-45df-b12d-7d4f6abb1cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:56.093201 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c7039-c1e1-45df-b12d-7d4f6abb1cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:56.093326 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c7039-c1e1-45df-b12d-7d4f6abb1cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:56.093708 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c7039-c1e1-45df-b12d-7d4f6abb1cd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:56.094437 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c7039-c1e1-45df-b12d-7d4f6abb1cd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:56.096598 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c7039-c1e1-45df-b12d-7d4f6abb1cd6 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:56.117186 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:56.176007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 878/1760] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:20:56 2022] POST /identity/v3/projects => generated 361 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:56.181342 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216e97b-be4d-48c7-b761-a45674d27143 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:56.194924 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216e97b-be4d-48c7-b761-a45674d27143 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:56.304455 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3fe55-36e3-49ab-a38d-75f348ba815b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:56.305270 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3fe55-36e3-49ab-a38d-75f348ba815b admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:56.305737 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3fe55-36e3-49ab-a38d-75f348ba815b admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:56.306208 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3fe55-36e3-49ab-a38d-75f348ba815b admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:56.307430 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3fe55-36e3-49ab-a38d-75f348ba815b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:56.318255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3fe55-36e3-49ab-a38d-75f348ba815b admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:56.381905 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 883/1761] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:20:56 2022] POST /identity/v3/users => generated 281 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:56.383530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c7039-c1e1-45df-b12d-7d4f6abb1cd6 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:56.401871 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c7039-c1e1-45df-b12d-7d4f6abb1cd6 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:56.427096 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b809153-9a05-4b48-9d5e-36a461e74b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:56.428327 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b809153-9a05-4b48-9d5e-36a461e74b4b admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:56.429382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b809153-9a05-4b48-9d5e-36a461e74b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:56.429752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b809153-9a05-4b48-9d5e-36a461e74b4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:56.430421 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b809153-9a05-4b48-9d5e-36a461e74b4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:56.434364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b809153-9a05-4b48-9d5e-36a461e74b4b admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:56.452407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 879/1762] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:20:56.456851 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3fe55-36e3-49ab-a38d-75f348ba815b admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:56.468453 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3fe55-36e3-49ab-a38d-75f348ba815b admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:56.474800 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7307de8-c9b3-49fe-a8f8-358716d58586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:56.475491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7307de8-c9b3-49fe-a8f8-358716d58586 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:56.476469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7307de8-c9b3-49fe-a8f8-358716d58586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:56.476801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7307de8-c9b3-49fe-a8f8-358716d58586 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:56.477750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7307de8-c9b3-49fe-a8f8-358716d58586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:56.479364 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7307de8-c9b3-49fe-a8f8-358716d58586 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:56.497750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 884/1763] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:20:56.499431 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b809153-9a05-4b48-9d5e-36a461e74b4b admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:56.527876 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b809153-9a05-4b48-9d5e-36a461e74b4b admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:56.552728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d107d6-e341-4b57-ab96-ee01932254d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:56.554032 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d107d6-e341-4b57-ab96-ee01932254d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:56.554652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d107d6-e341-4b57-ab96-ee01932254d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:56.555307 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d107d6-e341-4b57-ab96-ee01932254d1 admin admin] PATH_INFO: `/v3/projects/d82f635e0dea464682c2195ea945a689/users/71fb5b4350f8424e98d73b6e2051a4e5/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:56.580698 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d107d6-e341-4b57-ab96-ee01932254d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d82f635e0dea464682c2195ea945a689, user_id=71fb5b4350f8424e98d73b6e2051a4e5, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:56.584386 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d107d6-e341-4b57-ab96-ee01932254d1 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:56.602901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 880/1764] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:20:56 2022] PUT /identity/v3/projects/d82f635e0dea464682c2195ea945a689/users/71fb5b4350f8424e98d73b6e2051a4e5/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:56.613405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204563c-0fa6-4d90-9fe4-77b5f67f2331 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:56.613808 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204563c-0fa6-4d90-9fe4-77b5f67f2331 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:56.614175 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204563c-0fa6-4d90-9fe4-77b5f67f2331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:56.678519 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-9204563c-0fa6-4d90-9fe4-77b5f67f2331 None None] MFA Rules not processed for user `71fb5b4350f8424e98d73b6e2051a4e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:56.680261 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-9204563c-0fa6-4d90-9fe4-77b5f67f2331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:56.792852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 885/1765] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:56.797384 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d107d6-e341-4b57-ab96-ee01932254d1 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:56.879282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d107d6-e341-4b57-ab96-ee01932254d1 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:56.885612 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f17b2-ac84-489f-9fc9-459d3e61d71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:56.886231 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f17b2-ac84-489f-9fc9-459d3e61d71f admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:56.886512 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f17b2-ac84-489f-9fc9-459d3e61d71f admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:56.886787 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f17b2-ac84-489f-9fc9-459d3e61d71f admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:56.887458 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117f17b2-ac84-489f-9fc9-459d3e61d71f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:56.889367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117f17b2-ac84-489f-9fc9-459d3e61d71f admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:56.918234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 881/1766] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:20:56 2022] POST /identity/v3/projects => generated 365 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:56.922147 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204563c-0fa6-4d90-9fe4-77b5f67f2331 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:56.934016 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204563c-0fa6-4d90-9fe4-77b5f67f2331 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:56.968583 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f17b2-ac84-489f-9fc9-459d3e61d71f admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:56.970812 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-117f17b2-ac84-489f-9fc9-459d3e61d71f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:57.025866 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:57.044430 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c218b115-f902-43d4-949c-1cd28916840d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:57.045716 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218b115-f902-43d4-949c-1cd28916840d admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:57.046214 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218b115-f902-43d4-949c-1cd28916840d admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:57.046934 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218b115-f902-43d4-949c-1cd28916840d admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:57.049339 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c218b115-f902-43d4-949c-1cd28916840d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:57.051589 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c218b115-f902-43d4-949c-1cd28916840d admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:57.093883 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 886/1767] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:20:56 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:20:57.097764 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c218b115-f902-43d4-949c-1cd28916840d admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:57.099998 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f17b2-ac84-489f-9fc9-459d3e61d71f admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:57.104170 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb2d653-bb09-4dd5-bcaf-782f0b95249a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:57.105045 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2d653-bb09-4dd5-bcaf-782f0b95249a service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:57.105136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2d653-bb09-4dd5-bcaf-782f0b95249a service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:57.105341 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2d653-bb09-4dd5-bcaf-782f0b95249a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:57.107382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-edb2d653-bb09-4dd5-bcaf-782f0b95249a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:57.117420 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c218b115-f902-43d4-949c-1cd28916840d admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:57.127628 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb2d653-bb09-4dd5-bcaf-782f0b95249a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:57.129092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb2d653-bb09-4dd5-bcaf-782f0b95249a service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:57.138003 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0de6a-9998-494c-8aad-20b44c8df80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:57.138638 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0de6a-9998-494c-8aad-20b44c8df80c admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:57.138926 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0de6a-9998-494c-8aad-20b44c8df80c admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:57.139428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0de6a-9998-494c-8aad-20b44c8df80c admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:57.142994 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b0de6a-9998-494c-8aad-20b44c8df80c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:57.145499 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 882/1768] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:57.145832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b0de6a-9998-494c-8aad-20b44c8df80c admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:57.161339 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 887/1769] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:20:57.172470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb2d653-bb09-4dd5-bcaf-782f0b95249a service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:57.202273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb2d653-bb09-4dd5-bcaf-782f0b95249a service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:57.213869 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7d592-0c5e-47b0-b994-c1ec8e1e293d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:57.213869 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7d592-0c5e-47b0-b994-c1ec8e1e293d admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:57.217281 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7d592-0c5e-47b0-b994-c1ec8e1e293d admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:57.217669 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7d592-0c5e-47b0-b994-c1ec8e1e293d admin admin] PATH_INFO: `/v3/projects/6f8011223226432080f465c6f1b1f205/users/ace2cccde2c04be8890c666d06d8fb5b/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:57.265775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f7d592-0c5e-47b0-b994-c1ec8e1e293d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f8011223226432080f465c6f1b1f205, user_id=ace2cccde2c04be8890c666d06d8fb5b, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:57.272558 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f7d592-0c5e-47b0-b994-c1ec8e1e293d admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:57.298766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 883/1770] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:20:57 2022] PUT /identity/v3/projects/6f8011223226432080f465c6f1b1f205/users/ace2cccde2c04be8890c666d06d8fb5b/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:57.300117 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0de6a-9998-494c-8aad-20b44c8df80c admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:57.313261 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0de6a-9998-494c-8aad-20b44c8df80c admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:57.455067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1340d5-60eb-4776-a5eb-7d7ff4988f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:57.455715 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1340d5-60eb-4776-a5eb-7d7ff4988f25 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:57.455999 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1340d5-60eb-4776-a5eb-7d7ff4988f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:57.456277 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1340d5-60eb-4776-a5eb-7d7ff4988f25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:57.456945 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1340d5-60eb-4776-a5eb-7d7ff4988f25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:57.475278 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1340d5-60eb-4776-a5eb-7d7ff4988f25 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:57.489414 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 888/1771] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:20:57.493679 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7d592-0c5e-47b0-b994-c1ec8e1e293d admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:57.505070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7d592-0c5e-47b0-b994-c1ec8e1e293d admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:57.512087 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f567f8-a3cf-4984-9ac3-d0b3fadb92bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:57.512778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f567f8-a3cf-4984-9ac3-d0b3fadb92bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:57.513049 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f567f8-a3cf-4984-9ac3-d0b3fadb92bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:57.513332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f567f8-a3cf-4984-9ac3-d0b3fadb92bb admin admin] PATH_INFO: `/v3/projects/6f8011223226432080f465c6f1b1f205/users/ace2cccde2c04be8890c666d06d8fb5b/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:57.515343 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f567f8-a3cf-4984-9ac3-d0b3fadb92bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f8011223226432080f465c6f1b1f205, user_id=ace2cccde2c04be8890c666d06d8fb5b, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:57.518068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f567f8-a3cf-4984-9ac3-d0b3fadb92bb admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:57.526163 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:57.544102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 884/1772] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:20:57 2022] PUT /identity/v3/projects/6f8011223226432080f465c6f1b1f205/users/ace2cccde2c04be8890c666d06d8fb5b/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:57.557282 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47234b-3754-4b0b-a0b0-04fc08ed95b6 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:57.560652 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47234b-3754-4b0b-a0b0-04fc08ed95b6 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:57.561118 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47234b-3754-4b0b-a0b0-04fc08ed95b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:57.651510 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-ce47234b-3754-4b0b-a0b0-04fc08ed95b6 None None] MFA Rules not processed for user `ace2cccde2c04be8890c666d06d8fb5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:57.661504 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-ce47234b-3754-4b0b-a0b0-04fc08ed95b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:57.914518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 889/1773] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:57 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:57.923345 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f567f8-a3cf-4984-9ac3-d0b3fadb92bb admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:57.925481 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-a0f567f8-a3cf-4984-9ac3-d0b3fadb92bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:58.047709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f567f8-a3cf-4984-9ac3-d0b3fadb92bb admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:58.056584 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a5589-9deb-4abc-90ec-aab7f4fb6ae1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:58.057340 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a5589-9deb-4abc-90ec-aab7f4fb6ae1 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:58.057625 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a5589-9deb-4abc-90ec-aab7f4fb6ae1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:58.057902 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a5589-9deb-4abc-90ec-aab7f4fb6ae1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:58.079360 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a5589-9deb-4abc-90ec-aab7f4fb6ae1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:58.089682 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a5589-9deb-4abc-90ec-aab7f4fb6ae1 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:58.104842 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 885/1774] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:57 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:20:58.732100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f1c1d-03be-4cf9-99bc-7e037f0f9a9c None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:58.732391 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f1c1d-03be-4cf9-99bc-7e037f0f9a9c None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:58.732665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f1c1d-03be-4cf9-99bc-7e037f0f9a9c None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:58.734351 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 890/1775] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 31 00:20:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:20:58.781873 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7daac84-ba5e-48ed-87d3-151f9807f42d None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:58.782791 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7daac84-ba5e-48ed-87d3-151f9807f42d None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:58.783253 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7daac84-ba5e-48ed-87d3-151f9807f42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:58.815278 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-c7daac84-ba5e-48ed-87d3-151f9807f42d None None] MFA Rules not processed for user `3aa53ae5a30146f69d7d6eefe4290dba`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:58.816798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c7daac84-ba5e-48ed-87d3-151f9807f42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:58.912797 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f1c1d-03be-4cf9-99bc-7e037f0f9a9c None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:58.948113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 886/1776] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 31 00:20:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:59.007532 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:59.031011 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f1c1d-03be-4cf9-99bc-7e037f0f9a9c None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:59.039861 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060077bc-6cc3-4d93-8297-d8d97812919e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:59.040657 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060077bc-6cc3-4d93-8297-d8d97812919e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:59.040937 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060077bc-6cc3-4d93-8297-d8d97812919e admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:59.041323 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060077bc-6cc3-4d93-8297-d8d97812919e admin admin] PATH_INFO: `/v3/users/71fb5b4350f8424e98d73b6e2051a4e5` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:59.043268 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060077bc-6cc3-4d93-8297-d8d97812919e admin admin] RBAC: Authorizing `identity:delete_user(user_id=71fb5b4350f8424e98d73b6e2051a4e5)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:59.046577 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060077bc-6cc3-4d93-8297-d8d97812919e admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:59.107761 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-060077bc-6cc3-4d93-8297-d8d97812919e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71fb5b4350f8424e98d73b6e2051a4e5'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:59.118312 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-060077bc-6cc3-4d93-8297-d8d97812919e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71fb5b4350f8424e98d73b6e2051a4e5'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:59.126922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-060077bc-6cc3-4d93-8297-d8d97812919e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71fb5b4350f8424e98d73b6e2051a4e5'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:59.127729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-060077bc-6cc3-4d93-8297-d8d97812919e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71fb5b4350f8424e98d73b6e2051a4e5'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:59.128414 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-060077bc-6cc3-4d93-8297-d8d97812919e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71fb5b4350f8424e98d73b6e2051a4e5'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:59.136027 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7daac84-ba5e-48ed-87d3-151f9807f42d None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:59.137414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c7daac84-ba5e-48ed-87d3-151f9807f42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:59.148830 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 891/1777] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:20:58 2022] DELETE /identity/v3/users/71fb5b4350f8424e98d73b6e2051a4e5 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:59.152969 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63753c2-5f2e-4472-a000-acbcb7688094 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:59.153260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63753c2-5f2e-4472-a000-acbcb7688094 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:59.153536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63753c2-5f2e-4472-a000-acbcb7688094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:59.155122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7daac84-ba5e-48ed-87d3-151f9807f42d None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:59.172360 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-c63753c2-5f2e-4472-a000-acbcb7688094 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:20:59.173558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-c63753c2-5f2e-4472-a000-acbcb7688094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:59.221467 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aca0f0-4df3-4529-9937-50e463338073 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:59.222089 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aca0f0-4df3-4529-9937-50e463338073 service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:59.222367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aca0f0-4df3-4529-9937-50e463338073 service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:59.222638 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aca0f0-4df3-4529-9937-50e463338073 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:59.224393 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-71aca0f0-4df3-4529-9937-50e463338073 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:59.235962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aca0f0-4df3-4529-9937-50e463338073 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:59.237531 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aca0f0-4df3-4529-9937-50e463338073 service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:59.239044 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 892/1778] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:20:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:20:59.249208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 887/1779] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:20:59.249574 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63753c2-5f2e-4472-a000-acbcb7688094 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:59.251121 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-c63753c2-5f2e-4472-a000-acbcb7688094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:59.319368 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63753c2-5f2e-4472-a000-acbcb7688094 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:59.330809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2808fae-d04f-4acb-ba14-1f80d6139a44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:59.331790 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2808fae-d04f-4acb-ba14-1f80d6139a44 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:59.332226 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2808fae-d04f-4acb-ba14-1f80d6139a44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:59.332593 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2808fae-d04f-4acb-ba14-1f80d6139a44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:59.343162 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2808fae-d04f-4acb-ba14-1f80d6139a44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:59.344656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2808fae-d04f-4acb-ba14-1f80d6139a44 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:59.354580 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 893/1780] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:20:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:20:59.723890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aca0f0-4df3-4529-9937-50e463338073 service nova] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:59.731922 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-71aca0f0-4df3-4529-9937-50e463338073 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:20:59.757446 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aca0f0-4df3-4529-9937-50e463338073 service nova] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:59.779001 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f54a7-b367-4757-b094-5c993b1c7255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:20:59.780060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f54a7-b367-4757-b094-5c993b1c7255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:20:59.780414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f54a7-b367-4757-b094-5c993b1c7255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:20:59.780752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f54a7-b367-4757-b094-5c993b1c7255 admin admin] PATH_INFO: `/v3/projects/d82f635e0dea464682c2195ea945a689` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:20:59.781998 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f54a7-b367-4757-b094-5c993b1c7255 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d82f635e0dea464682c2195ea945a689)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:20:59.784214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f54a7-b367-4757-b094-5c993b1c7255 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:20:59.813210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:20:59.864002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f78f54a7-b367-4757-b094-5c993b1c7255 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd82f635e0dea464682c2195ea945a689', 'request_id': 'req-f78f54a7-b367-4757-b094-5c993b1c7255'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:59.880767 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f78f54a7-b367-4757-b094-5c993b1c7255 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd82f635e0dea464682c2195ea945a689', 'request_id': 'req-f78f54a7-b367-4757-b094-5c993b1c7255'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:59.887733 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f78f54a7-b367-4757-b094-5c993b1c7255 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd82f635e0dea464682c2195ea945a689', 'request_id': 'req-f78f54a7-b367-4757-b094-5c993b1c7255'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:59.900763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f78f54a7-b367-4757-b094-5c993b1c7255 admin admin] The token cache is being invalidate because project d82f635e0dea464682c2195ea945a689 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:20:59.901067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f78f54a7-b367-4757-b094-5c993b1c7255 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:59.901818 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f78f54a7-b367-4757-b094-5c993b1c7255 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:20:59.903539 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 888/1781] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:20:59 2022] DELETE /identity/v3/projects/d82f635e0dea464682c2195ea945a689 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:20:59.907788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2808fae-d04f-4acb-ba14-1f80d6139a44 service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:20:59.919259 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2808fae-d04f-4acb-ba14-1f80d6139a44 service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:20:59.933534 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:00.021601 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8892dc3b-86b7-4fe8-8c9d-2558352b114c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:00.022362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892dc3b-86b7-4fe8-8c9d-2558352b114c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:00.022561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892dc3b-86b7-4fe8-8c9d-2558352b114c admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:00.023068 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892dc3b-86b7-4fe8-8c9d-2558352b114c admin admin] PATH_INFO: `/v3/users/ace2cccde2c04be8890c666d06d8fb5b` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:00.024478 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8892dc3b-86b7-4fe8-8c9d-2558352b114c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ace2cccde2c04be8890c666d06d8fb5b)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:00.026585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8892dc3b-86b7-4fe8-8c9d-2558352b114c admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:00.088119 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8892dc3b-86b7-4fe8-8c9d-2558352b114c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ace2cccde2c04be8890c666d06d8fb5b'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:00.096243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8892dc3b-86b7-4fe8-8c9d-2558352b114c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ace2cccde2c04be8890c666d06d8fb5b'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:00.104165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8892dc3b-86b7-4fe8-8c9d-2558352b114c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ace2cccde2c04be8890c666d06d8fb5b'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:00.104820 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8892dc3b-86b7-4fe8-8c9d-2558352b114c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ace2cccde2c04be8890c666d06d8fb5b'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:00.105425 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8892dc3b-86b7-4fe8-8c9d-2558352b114c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ace2cccde2c04be8890c666d06d8fb5b'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:00.123527 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 894/1782] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:20:59 2022] DELETE /identity/v3/users/ace2cccde2c04be8890c666d06d8fb5b => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:00.490884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f54a7-b367-4757-b094-5c993b1c7255 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:00.493423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f78f54a7-b367-4757-b094-5c993b1c7255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:00.526293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f54a7-b367-4757-b094-5c993b1c7255 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:00.639293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4e018-04bf-4306-b995-6719f90421c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:00.640195 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4e018-04bf-4306-b995-6719f90421c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:00.640647 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4e018-04bf-4306-b995-6719f90421c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:00.640975 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4e018-04bf-4306-b995-6719f90421c9 admin admin] PATH_INFO: `/v3/projects/6f8011223226432080f465c6f1b1f205` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:00.642392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c4e018-04bf-4306-b995-6719f90421c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f8011223226432080f465c6f1b1f205)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:00.644643 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c4e018-04bf-4306-b995-6719f90421c9 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:00.747360 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-92c4e018-04bf-4306-b995-6719f90421c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f8011223226432080f465c6f1b1f205', 'request_id': 'req-92c4e018-04bf-4306-b995-6719f90421c9'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:00.761631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-92c4e018-04bf-4306-b995-6719f90421c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f8011223226432080f465c6f1b1f205', 'request_id': 'req-92c4e018-04bf-4306-b995-6719f90421c9'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:00.772164 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-92c4e018-04bf-4306-b995-6719f90421c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f8011223226432080f465c6f1b1f205', 'request_id': 'req-92c4e018-04bf-4306-b995-6719f90421c9'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:00.796208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-92c4e018-04bf-4306-b995-6719f90421c9 admin admin] The token cache is being invalidate because project 6f8011223226432080f465c6f1b1f205 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:21:00.797567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-92c4e018-04bf-4306-b995-6719f90421c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:00.798453 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-92c4e018-04bf-4306-b995-6719f90421c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:00.800620 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 889/1783] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:21:00 2022] DELETE /identity/v3/projects/6f8011223226432080f465c6f1b1f205 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:00.806091 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e6521-fa9c-41a9-95f3-4109886d714a None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:00.806449 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e6521-fa9c-41a9-95f3-4109886d714a None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:00.806794 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e6521-fa9c-41a9-95f3-4109886d714a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:00.828969 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-4b6e6521-fa9c-41a9-95f3-4109886d714a None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:00.830300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4b6e6521-fa9c-41a9-95f3-4109886d714a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:00.938219 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 895/1784] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:21:00.946332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd2198-110d-448b-bd3e-3fcaf3c64a6a None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:00.946554 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd2198-110d-448b-bd3e-3fcaf3c64a6a None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:00.946826 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd2198-110d-448b-bd3e-3fcaf3c64a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:00.969532 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-a6fd2198-110d-448b-bd3e-3fcaf3c64a6a None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:00.970751 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-a6fd2198-110d-448b-bd3e-3fcaf3c64a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:00.980240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 890/1785] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:00.984495 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e6521-fa9c-41a9-95f3-4109886d714a None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:00.994657 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e6521-fa9c-41a9-95f3-4109886d714a None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:00.999326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce3e91-14e4-4f49-8151-017a06e180db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:00.999930 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce3e91-14e4-4f49-8151-017a06e180db admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:01.000224 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce3e91-14e4-4f49-8151-017a06e180db admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:01.000475 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce3e91-14e4-4f49-8151-017a06e180db admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:01.001118 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce3e91-14e4-4f49-8151-017a06e180db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:21:01.001417 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce3e91-14e4-4f49-8151-017a06e180db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:01.003006 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce3e91-14e4-4f49-8151-017a06e180db admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:01.015311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 896/1786] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:21:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:21:01.019931 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd2198-110d-448b-bd3e-3fcaf3c64a6a None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:01.043449 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd2198-110d-448b-bd3e-3fcaf3c64a6a None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:01.051177 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769725dc-a2c5-4197-98ae-7e9f25fadabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:01.051938 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769725dc-a2c5-4197-98ae-7e9f25fadabb admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:01.052235 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769725dc-a2c5-4197-98ae-7e9f25fadabb admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:01.052584 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769725dc-a2c5-4197-98ae-7e9f25fadabb admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:01.053283 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769725dc-a2c5-4197-98ae-7e9f25fadabb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:01.055804 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769725dc-a2c5-4197-98ae-7e9f25fadabb admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:01.098676 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 891/1787] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:21:01 2022] POST /identity/v3/projects => generated 385 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:01.105394 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce3e91-14e4-4f49-8151-017a06e180db admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:01.126761 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce3e91-14e4-4f49-8151-017a06e180db admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:01.151407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:01.233025 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e246c3-2a5b-4231-af68-c37903a297ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:01.233660 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e246c3-2a5b-4231-af68-c37903a297ef admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:01.233997 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e246c3-2a5b-4231-af68-c37903a297ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:01.234275 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e246c3-2a5b-4231-af68-c37903a297ef admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:01.235084 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e246c3-2a5b-4231-af68-c37903a297ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:01.237229 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e246c3-2a5b-4231-af68-c37903a297ef admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:01.272038 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 897/1788] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:21:01 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:21:01.277377 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769725dc-a2c5-4197-98ae-7e9f25fadabb admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:01.284817 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:01.294308 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769725dc-a2c5-4197-98ae-7e9f25fadabb admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:01.303270 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32febde4-6c7b-4eba-83fe-d83b76dd7b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:01.304381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32febde4-6c7b-4eba-83fe-d83b76dd7b3a admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:01.304724 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32febde4-6c7b-4eba-83fe-d83b76dd7b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:01.305286 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32febde4-6c7b-4eba-83fe-d83b76dd7b3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:01.305994 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32febde4-6c7b-4eba-83fe-d83b76dd7b3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:01.307965 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32febde4-6c7b-4eba-83fe-d83b76dd7b3a admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:01.320803 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 892/1789] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:01.325058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e246c3-2a5b-4231-af68-c37903a297ef admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:01.336165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e246c3-2a5b-4231-af68-c37903a297ef admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:01.341953 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8809148-b909-4f05-9a03-219f4ea21f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:01.342646 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8809148-b909-4f05-9a03-219f4ea21f2d admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:01.342994 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8809148-b909-4f05-9a03-219f4ea21f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:01.343374 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8809148-b909-4f05-9a03-219f4ea21f2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:01.344095 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8809148-b909-4f05-9a03-219f4ea21f2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:01.345954 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8809148-b909-4f05-9a03-219f4ea21f2d admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:01.365105 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 898/1790] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:01.369755 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32febde4-6c7b-4eba-83fe-d83b76dd7b3a admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:01.382001 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32febde4-6c7b-4eba-83fe-d83b76dd7b3a admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:01.388026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411d7e4-f540-4a04-97a7-fbeaf5fcd687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:01.388822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411d7e4-f540-4a04-97a7-fbeaf5fcd687 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:01.389141 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411d7e4-f540-4a04-97a7-fbeaf5fcd687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:01.389380 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411d7e4-f540-4a04-97a7-fbeaf5fcd687 admin admin] PATH_INFO: `/v3/projects/5ab67b94d9c841db93de438baa205659/users/f6408d8c093841a2a0f1ac2c1eaf7fe9/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:01.428271 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d411d7e4-f540-4a04-97a7-fbeaf5fcd687 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ab67b94d9c841db93de438baa205659, user_id=f6408d8c093841a2a0f1ac2c1eaf7fe9, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:01.434720 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d411d7e4-f540-4a04-97a7-fbeaf5fcd687 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:01.468471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 893/1791] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:01 2022] PUT /identity/v3/projects/5ab67b94d9c841db93de438baa205659/users/f6408d8c093841a2a0f1ac2c1eaf7fe9/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:01.480569 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6555de-2114-4031-b79d-dea8c94e8827 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:01.480904 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6555de-2114-4031-b79d-dea8c94e8827 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:01.481180 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6555de-2114-4031-b79d-dea8c94e8827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:01.519343 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-db6555de-2114-4031-b79d-dea8c94e8827 None None] MFA Rules not processed for user `f6408d8c093841a2a0f1ac2c1eaf7fe9`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:01.521925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-db6555de-2114-4031-b79d-dea8c94e8827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:01.640680 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 899/1792] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:01.643156 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411d7e4-f540-4a04-97a7-fbeaf5fcd687 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:01.766876 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411d7e4-f540-4a04-97a7-fbeaf5fcd687 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:01.780116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f8a20-ed5b-41c5-a3ee-bbd13940cf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:01.780751 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f8a20-ed5b-41c5-a3ee-bbd13940cf3e admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:01.781033 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f8a20-ed5b-41c5-a3ee-bbd13940cf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:01.781304 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f8a20-ed5b-41c5-a3ee-bbd13940cf3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:01.781947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f8a20-ed5b-41c5-a3ee-bbd13940cf3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:01.783880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f8a20-ed5b-41c5-a3ee-bbd13940cf3e admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:01.900957 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 894/1793] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:21:01 2022] POST /identity/v3/projects => generated 385 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:01.912236 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6555de-2114-4031-b79d-dea8c94e8827 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:01.935790 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6555de-2114-4031-b79d-dea8c94e8827 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:02.048868 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c2c75-67c8-4a23-accd-ae256bc8af54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:02.049769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c2c75-67c8-4a23-accd-ae256bc8af54 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:02.050282 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c2c75-67c8-4a23-accd-ae256bc8af54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:02.050991 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c2c75-67c8-4a23-accd-ae256bc8af54 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:02.055196 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3c2c75-67c8-4a23-accd-ae256bc8af54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:02.061549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3c2c75-67c8-4a23-accd-ae256bc8af54 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:02.093101 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 900/1794] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:21:01 2022] POST /identity/v3/users => generated 299 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:02.097741 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f8a20-ed5b-41c5-a3ee-bbd13940cf3e admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:02.113705 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f8a20-ed5b-41c5-a3ee-bbd13940cf3e admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:02.137330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435ec44-884d-40ec-acd3-f13431baf609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:02.137330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435ec44-884d-40ec-acd3-f13431baf609 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:02.137330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435ec44-884d-40ec-acd3-f13431baf609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:02.137699 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435ec44-884d-40ec-acd3-f13431baf609 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:02.138068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1435ec44-884d-40ec-acd3-f13431baf609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:02.139730 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1435ec44-884d-40ec-acd3-f13431baf609 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:02.169105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 895/1795] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:02.169350 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c2c75-67c8-4a23-accd-ae256bc8af54 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:02.189478 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c2c75-67c8-4a23-accd-ae256bc8af54 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:02.200726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53275918-5e39-4fff-a2e2-a553fb53d32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:02.200726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53275918-5e39-4fff-a2e2-a553fb53d32d admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:02.200726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53275918-5e39-4fff-a2e2-a553fb53d32d admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:02.214750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53275918-5e39-4fff-a2e2-a553fb53d32d admin admin] PATH_INFO: `/v3/projects/a1a3fc654a2a4e7f94ace8cb6f2b309f/users/0c8f9a2b69124435a97c6d7aa8a2e101/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:02.291355 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53275918-5e39-4fff-a2e2-a553fb53d32d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a3fc654a2a4e7f94ace8cb6f2b309f, user_id=0c8f9a2b69124435a97c6d7aa8a2e101, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:02.291355 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53275918-5e39-4fff-a2e2-a553fb53d32d admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:02.291355 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:02.322035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 901/1796] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:02 2022] PUT /identity/v3/projects/a1a3fc654a2a4e7f94ace8cb6f2b309f/users/0c8f9a2b69124435a97c6d7aa8a2e101/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:02.326437 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435ec44-884d-40ec-acd3-f13431baf609 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:02.346138 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435ec44-884d-40ec-acd3-f13431baf609 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:02.554787 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995d9fb-cad8-41c0-b286-403cafbeceeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:02.555808 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995d9fb-cad8-41c0-b286-403cafbeceeb admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:02.556298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995d9fb-cad8-41c0-b286-403cafbeceeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:02.556715 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995d9fb-cad8-41c0-b286-403cafbeceeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:02.557804 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a995d9fb-cad8-41c0-b286-403cafbeceeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:02.561142 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a995d9fb-cad8-41c0-b286-403cafbeceeb admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:02.579441 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 896/1797] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:02.583467 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53275918-5e39-4fff-a2e2-a553fb53d32d admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:02.601664 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53275918-5e39-4fff-a2e2-a553fb53d32d admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:02.610952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2a16e-c396-4b5c-b4ae-091f19d5a154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:02.612439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2a16e-c396-4b5c-b4ae-091f19d5a154 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:02.612725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2a16e-c396-4b5c-b4ae-091f19d5a154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:02.612998 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2a16e-c396-4b5c-b4ae-091f19d5a154 admin admin] PATH_INFO: `/v3/projects/a1a3fc654a2a4e7f94ace8cb6f2b309f/users/0c8f9a2b69124435a97c6d7aa8a2e101/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:02.615875 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2a16e-c396-4b5c-b4ae-091f19d5a154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a3fc654a2a4e7f94ace8cb6f2b309f, user_id=0c8f9a2b69124435a97c6d7aa8a2e101, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:02.618152 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2a16e-c396-4b5c-b4ae-091f19d5a154 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:02.635968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 902/1798] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:02 2022] PUT /identity/v3/projects/a1a3fc654a2a4e7f94ace8cb6f2b309f/users/0c8f9a2b69124435a97c6d7aa8a2e101/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:02.651913 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c9deb-222b-4f23-8d5a-5dc974d87dd3 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:02.652355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c9deb-222b-4f23-8d5a-5dc974d87dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:02.653103 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c9deb-222b-4f23-8d5a-5dc974d87dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:02.736834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-819c9deb-222b-4f23-8d5a-5dc974d87dd3 None None] MFA Rules not processed for user `0c8f9a2b69124435a97c6d7aa8a2e101`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:02.745576 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-819c9deb-222b-4f23-8d5a-5dc974d87dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:02.824454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:02.943219 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 897/1799] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:02 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:02.956551 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2a16e-c396-4b5c-b4ae-091f19d5a154 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:02.958991 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-20f2a16e-c396-4b5c-b4ae-091f19d5a154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:03.180930 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2a16e-c396-4b5c-b4ae-091f19d5a154 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:03.192794 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838ac59-edb9-40d1-b3db-179549dd79c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:03.193437 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838ac59-edb9-40d1-b3db-179549dd79c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:03.193722 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838ac59-edb9-40d1-b3db-179549dd79c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:03.193996 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838ac59-edb9-40d1-b3db-179549dd79c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:03.207873 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838ac59-edb9-40d1-b3db-179549dd79c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:03.209511 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838ac59-edb9-40d1-b3db-179549dd79c3 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:03.247392 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 903/1800] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:02 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:21:03.369182 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c9deb-222b-4f23-8d5a-5dc974d87dd3 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:03.599246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c9deb-222b-4f23-8d5a-5dc974d87dd3 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:03.615228 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d475d99-cccc-4575-8786-a7ffb7af8317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:03.617828 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d475d99-cccc-4575-8786-a7ffb7af8317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:03.621012 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d475d99-cccc-4575-8786-a7ffb7af8317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:03.621687 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d475d99-cccc-4575-8786-a7ffb7af8317 admin admin] PATH_INFO: `/v3/users/f6408d8c093841a2a0f1ac2c1eaf7fe9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:03.624750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d475d99-cccc-4575-8786-a7ffb7af8317 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6408d8c093841a2a0f1ac2c1eaf7fe9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:03.636106 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d475d99-cccc-4575-8786-a7ffb7af8317 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:03.804715 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-3d475d99-cccc-4575-8786-a7ffb7af8317 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6408d8c093841a2a0f1ac2c1eaf7fe9'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:03.889305 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-3d475d99-cccc-4575-8786-a7ffb7af8317 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6408d8c093841a2a0f1ac2c1eaf7fe9'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:03.909218 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-3d475d99-cccc-4575-8786-a7ffb7af8317 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6408d8c093841a2a0f1ac2c1eaf7fe9'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:03.940337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-3d475d99-cccc-4575-8786-a7ffb7af8317 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6408d8c093841a2a0f1ac2c1eaf7fe9'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:03.941131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-3d475d99-cccc-4575-8786-a7ffb7af8317 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6408d8c093841a2a0f1ac2c1eaf7fe9'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:04.019343 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 898/1801] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:21:03 2022] DELETE /identity/v3/users/f6408d8c093841a2a0f1ac2c1eaf7fe9 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:04.020505 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96defe3-2987-4d99-97c2-607a5e2cc177 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:04.021431 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96defe3-2987-4d99-97c2-607a5e2cc177 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:04.022615 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96defe3-2987-4d99-97c2-607a5e2cc177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:04.126189 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-a96defe3-2987-4d99-97c2-607a5e2cc177 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:04.134302 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a96defe3-2987-4d99-97c2-607a5e2cc177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:04.401886 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 904/1802] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:04.499563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d475d99-cccc-4575-8786-a7ffb7af8317 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:04.501126 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-3d475d99-cccc-4575-8786-a7ffb7af8317 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:04.651310 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:04.675749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d475d99-cccc-4575-8786-a7ffb7af8317 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:04.683408 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68534ff4-d52c-4584-b522-0f6579f33b78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:04.684189 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68534ff4-d52c-4584-b522-0f6579f33b78 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:04.684568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68534ff4-d52c-4584-b522-0f6579f33b78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:04.684942 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68534ff4-d52c-4584-b522-0f6579f33b78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:04.707662 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68534ff4-d52c-4584-b522-0f6579f33b78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:04.709140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68534ff4-d52c-4584-b522-0f6579f33b78 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:04.722902 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 899/1803] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:21:04.978552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96defe3-2987-4d99-97c2-607a5e2cc177 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:04.987423 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a96defe3-2987-4d99-97c2-607a5e2cc177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:05.015750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96defe3-2987-4d99-97c2-607a5e2cc177 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:05.021640 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b60ae-4d91-41fd-a07d-5e4b7858ba58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:05.022521 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b60ae-4d91-41fd-a07d-5e4b7858ba58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:05.023136 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b60ae-4d91-41fd-a07d-5e4b7858ba58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:05.023806 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b60ae-4d91-41fd-a07d-5e4b7858ba58 admin admin] PATH_INFO: `/v3/users/77f2e82d6d5e437baf931c4b5e78c551` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:05.026416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b60ae-4d91-41fd-a07d-5e4b7858ba58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77f2e82d6d5e437baf931c4b5e78c551)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:05.028809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b60ae-4d91-41fd-a07d-5e4b7858ba58 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:05.159325 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-992b60ae-4d91-41fd-a07d-5e4b7858ba58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77f2e82d6d5e437baf931c4b5e78c551'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:05.171961 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-992b60ae-4d91-41fd-a07d-5e4b7858ba58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77f2e82d6d5e437baf931c4b5e78c551'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:05.180878 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-992b60ae-4d91-41fd-a07d-5e4b7858ba58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77f2e82d6d5e437baf931c4b5e78c551'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:05.182040 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-992b60ae-4d91-41fd-a07d-5e4b7858ba58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77f2e82d6d5e437baf931c4b5e78c551'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:05.182840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-992b60ae-4d91-41fd-a07d-5e4b7858ba58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77f2e82d6d5e437baf931c4b5e78c551'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:05.209370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:05.229718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 905/1804] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:21:04 2022] DELETE /identity/v3/users/77f2e82d6d5e437baf931c4b5e78c551 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:21:05.234104 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458d084-52b1-4094-9a44-dd0e7a89ac49 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:05.234754 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458d084-52b1-4094-9a44-dd0e7a89ac49 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:05.234754 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458d084-52b1-4094-9a44-dd0e7a89ac49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:05.271454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-5458d084-52b1-4094-9a44-dd0e7a89ac49 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:05.273175 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-5458d084-52b1-4094-9a44-dd0e7a89ac49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:05.351798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b60ae-4d91-41fd-a07d-5e4b7858ba58 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:05.353878 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-992b60ae-4d91-41fd-a07d-5e4b7858ba58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:05.501451 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 900/1805] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:05.523569 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458d084-52b1-4094-9a44-dd0e7a89ac49 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:05.530149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-5458d084-52b1-4094-9a44-dd0e7a89ac49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:05.549961 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b60ae-4d91-41fd-a07d-5e4b7858ba58 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:05.562124 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c50a4c-c86d-4e94-82d1-75d1091b3c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:05.566210 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c50a4c-c86d-4e94-82d1-75d1091b3c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:05.566605 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c50a4c-c86d-4e94-82d1-75d1091b3c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:05.566994 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c50a4c-c86d-4e94-82d1-75d1091b3c47 admin admin] PATH_INFO: `/v3/projects/5ab67b94d9c841db93de438baa205659` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:05.569092 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c50a4c-c86d-4e94-82d1-75d1091b3c47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ab67b94d9c841db93de438baa205659)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:05.572487 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c50a4c-c86d-4e94-82d1-75d1091b3c47 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:05.651953 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458d084-52b1-4094-9a44-dd0e7a89ac49 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:05.699948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d4c50a4c-c86d-4e94-82d1-75d1091b3c47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ab67b94d9c841db93de438baa205659', 'request_id': 'req-d4c50a4c-c86d-4e94-82d1-75d1091b3c47'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:05.712710 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d4c50a4c-c86d-4e94-82d1-75d1091b3c47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ab67b94d9c841db93de438baa205659', 'request_id': 'req-d4c50a4c-c86d-4e94-82d1-75d1091b3c47'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:05.723076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d4c50a4c-c86d-4e94-82d1-75d1091b3c47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ab67b94d9c841db93de438baa205659', 'request_id': 'req-d4c50a4c-c86d-4e94-82d1-75d1091b3c47'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:05.738804 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff0e53-b5e7-439e-955c-5dafe7fbc627 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:05.739496 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff0e53-b5e7-439e-955c-5dafe7fbc627 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:05.739775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff0e53-b5e7-439e-955c-5dafe7fbc627 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:05.740050 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff0e53-b5e7-439e-955c-5dafe7fbc627 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:05.784217 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d4c50a4c-c86d-4e94-82d1-75d1091b3c47 admin admin] The token cache is being invalidate because project 5ab67b94d9c841db93de438baa205659 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:21:05.784217 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d4c50a4c-c86d-4e94-82d1-75d1091b3c47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:05.785516 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d4c50a4c-c86d-4e94-82d1-75d1091b3c47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:05.790827 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 906/1806] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:21:05 2022] DELETE /identity/v3/projects/5ab67b94d9c841db93de438baa205659 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:05.798403 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c50a4c-c86d-4e94-82d1-75d1091b3c47 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:05.800509 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-d4c50a4c-c86d-4e94-82d1-75d1091b3c47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:05.927880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff0e53-b5e7-439e-955c-5dafe7fbc627 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:05.930166 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff0e53-b5e7-439e-955c-5dafe7fbc627 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:05.947424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:05.975623 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c50a4c-c86d-4e94-82d1-75d1091b3c47 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:05.985873 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 901/1807] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:21:06.001696 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e9964-181f-4604-b511-813decd59fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:06.002861 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e9964-181f-4604-b511-813decd59fc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:06.002861 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e9964-181f-4604-b511-813decd59fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:06.003512 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e9964-181f-4604-b511-813decd59fc0 admin admin] PATH_INFO: `/v3/users/0c8f9a2b69124435a97c6d7aa8a2e101` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:06.017527 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e9964-181f-4604-b511-813decd59fc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c8f9a2b69124435a97c6d7aa8a2e101)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:06.020500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e9964-181f-4604-b511-813decd59fc0 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:06.134442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:06.199189 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff0e53-b5e7-439e-955c-5dafe7fbc627 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:06.200792 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-a5ff0e53-b5e7-439e-955c-5dafe7fbc627 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:06.204331 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-6a1e9964-181f-4604-b511-813decd59fc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c8f9a2b69124435a97c6d7aa8a2e101'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:06.222012 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff0e53-b5e7-439e-955c-5dafe7fbc627 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:06.227425 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-6a1e9964-181f-4604-b511-813decd59fc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c8f9a2b69124435a97c6d7aa8a2e101'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:06.237764 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-6a1e9964-181f-4604-b511-813decd59fc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c8f9a2b69124435a97c6d7aa8a2e101'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:06.238845 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-6a1e9964-181f-4604-b511-813decd59fc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c8f9a2b69124435a97c6d7aa8a2e101'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:06.239638 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-6a1e9964-181f-4604-b511-813decd59fc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c8f9a2b69124435a97c6d7aa8a2e101'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:06.252183 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efa5fb-c33f-4593-8ca8-02e31d988ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:06.252985 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efa5fb-c33f-4593-8ca8-02e31d988ffb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:06.253273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efa5fb-c33f-4593-8ca8-02e31d988ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:06.253547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efa5fb-c33f-4593-8ca8-02e31d988ffb admin admin] PATH_INFO: `/v3/users/4fdcfd233bfd4d6ca1bc16f14f069469` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:06.255895 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87efa5fb-c33f-4593-8ca8-02e31d988ffb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fdcfd233bfd4d6ca1bc16f14f069469)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:06.258891 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87efa5fb-c33f-4593-8ca8-02e31d988ffb admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:06.293637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 907/1808] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:21:05 2022] DELETE /identity/v3/users/0c8f9a2b69124435a97c6d7aa8a2e101 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:06.338153 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-87efa5fb-c33f-4593-8ca8-02e31d988ffb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fdcfd233bfd4d6ca1bc16f14f069469'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:06.354801 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-87efa5fb-c33f-4593-8ca8-02e31d988ffb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fdcfd233bfd4d6ca1bc16f14f069469'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:06.362854 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-87efa5fb-c33f-4593-8ca8-02e31d988ffb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fdcfd233bfd4d6ca1bc16f14f069469'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:06.373063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-87efa5fb-c33f-4593-8ca8-02e31d988ffb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fdcfd233bfd4d6ca1bc16f14f069469'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:06.373968 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-87efa5fb-c33f-4593-8ca8-02e31d988ffb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fdcfd233bfd4d6ca1bc16f14f069469'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:06.400036 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 902/1809] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:21:06 2022] DELETE /identity/v3/users/4fdcfd233bfd4d6ca1bc16f14f069469 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:21:06.406204 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ce0c1-5ff3-447f-8dd0-3a3619d5c18c None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:06.406550 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ce0c1-5ff3-447f-8dd0-3a3619d5c18c None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:06.407664 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ce0c1-5ff3-447f-8dd0-3a3619d5c18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:06.449741 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-df2ce0c1-5ff3-447f-8dd0-3a3619d5c18c None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:06.451294 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-df2ce0c1-5ff3-447f-8dd0-3a3619d5c18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:06.535404 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 908/1810] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:06.561741 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efa5fb-c33f-4593-8ca8-02e31d988ffb admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:06.564430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-87efa5fb-c33f-4593-8ca8-02e31d988ffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:06.617698 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ce0c1-5ff3-447f-8dd0-3a3619d5c18c None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:06.619610 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-df2ce0c1-5ff3-447f-8dd0-3a3619d5c18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:06.639400 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efa5fb-c33f-4593-8ca8-02e31d988ffb admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:06.643837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fae540-3ae1-42e7-90b3-79e17b11de58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:06.646661 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ce0c1-5ff3-447f-8dd0-3a3619d5c18c None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:06.651614 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fae540-3ae1-42e7-90b3-79e17b11de58 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:06.652112 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fae540-3ae1-42e7-90b3-79e17b11de58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:06.652536 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fae540-3ae1-42e7-90b3-79e17b11de58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:06.654625 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0416cf-3ead-486e-acba-169c8952b9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:06.655475 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0416cf-3ead-486e-acba-169c8952b9ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:06.664557 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fae540-3ae1-42e7-90b3-79e17b11de58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:06.665498 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0416cf-3ead-486e-acba-169c8952b9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:06.665999 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fae540-3ae1-42e7-90b3-79e17b11de58 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:06.666120 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0416cf-3ead-486e-acba-169c8952b9ed admin admin] PATH_INFO: `/v3/projects/bfec858a072145f3a7b7ac593280e7e0` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:06.667611 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0416cf-3ead-486e-acba-169c8952b9ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfec858a072145f3a7b7ac593280e7e0)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:06.670069 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0416cf-3ead-486e-acba-169c8952b9ed admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:06.676683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 903/1811] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:21:06.713806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fae540-3ae1-42e7-90b3-79e17b11de58 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:06.715551 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-d4fae540-3ae1-42e7-90b3-79e17b11de58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:06.740159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fae540-3ae1-42e7-90b3-79e17b11de58 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:06.752445 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b82f9e-7152-4df1-af29-716e3bedce70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:06.753193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b82f9e-7152-4df1-af29-716e3bedce70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:06.753593 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b82f9e-7152-4df1-af29-716e3bedce70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:06.753902 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b82f9e-7152-4df1-af29-716e3bedce70 admin admin] PATH_INFO: `/v3/projects/a1a3fc654a2a4e7f94ace8cb6f2b309f` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:06.755931 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b82f9e-7152-4df1-af29-716e3bedce70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1a3fc654a2a4e7f94ace8cb6f2b309f)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:06.758780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b82f9e-7152-4df1-af29-716e3bedce70 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:06.766727 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-1e0416cf-3ead-486e-acba-169c8952b9ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfec858a072145f3a7b7ac593280e7e0', 'request_id': 'req-1e0416cf-3ead-486e-acba-169c8952b9ed'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:06.775626 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-1e0416cf-3ead-486e-acba-169c8952b9ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfec858a072145f3a7b7ac593280e7e0', 'request_id': 'req-1e0416cf-3ead-486e-acba-169c8952b9ed'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:06.785286 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-1e0416cf-3ead-486e-acba-169c8952b9ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfec858a072145f3a7b7ac593280e7e0', 'request_id': 'req-1e0416cf-3ead-486e-acba-169c8952b9ed'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:06.805084 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-1e0416cf-3ead-486e-acba-169c8952b9ed admin admin] The token cache is being invalidate because project bfec858a072145f3a7b7ac593280e7e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:21:06.805381 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-1e0416cf-3ead-486e-acba-169c8952b9ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:06.806054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-1e0416cf-3ead-486e-acba-169c8952b9ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:06.811232 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 909/1812] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:21:06 2022] DELETE /identity/v3/projects/bfec858a072145f3a7b7ac593280e7e0 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:06.812671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638dad8-412c-4a55-9801-f9ef0e2a85cc None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:06.812953 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638dad8-412c-4a55-9801-f9ef0e2a85cc None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:06.813229 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638dad8-412c-4a55-9801-f9ef0e2a85cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:06.818625 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e1b82f9e-7152-4df1-af29-716e3bedce70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1a3fc654a2a4e7f94ace8cb6f2b309f', 'request_id': 'req-e1b82f9e-7152-4df1-af29-716e3bedce70'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:06.831848 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e1b82f9e-7152-4df1-af29-716e3bedce70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1a3fc654a2a4e7f94ace8cb6f2b309f', 'request_id': 'req-e1b82f9e-7152-4df1-af29-716e3bedce70'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:06.837945 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-e638dad8-412c-4a55-9801-f9ef0e2a85cc None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:06.839029 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:06.839417 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e638dad8-412c-4a55-9801-f9ef0e2a85cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:06.840934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e1b82f9e-7152-4df1-af29-716e3bedce70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1a3fc654a2a4e7f94ace8cb6f2b309f', 'request_id': 'req-e1b82f9e-7152-4df1-af29-716e3bedce70'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:06.857548 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e1b82f9e-7152-4df1-af29-716e3bedce70 admin admin] The token cache is being invalidate because project a1a3fc654a2a4e7f94ace8cb6f2b309f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:21:06.857946 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e1b82f9e-7152-4df1-af29-716e3bedce70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:06.859165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e1b82f9e-7152-4df1-af29-716e3bedce70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:06.866009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 904/1813] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:21:06 2022] DELETE /identity/v3/projects/a1a3fc654a2a4e7f94ace8cb6f2b309f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:06.867778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7418fd0-6cbc-45f6-888e-43661c4349e6 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:06.868169 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7418fd0-6cbc-45f6-888e-43661c4349e6 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:06.868536 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7418fd0-6cbc-45f6-888e-43661c4349e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:06.875759 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:06.898470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-c7418fd0-6cbc-45f6-888e-43661c4349e6 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:06.900438 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c7418fd0-6cbc-45f6-888e-43661c4349e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:06.927795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 910/1814] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:06.936101 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843262e-70b5-4295-a61e-90417c943c0a None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:06.936430 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843262e-70b5-4295-a61e-90417c943c0a None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:06.936715 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843262e-70b5-4295-a61e-90417c943c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:06.959380 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-5843262e-70b5-4295-a61e-90417c943c0a None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:06.961827 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-5843262e-70b5-4295-a61e-90417c943c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:06.975016 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 905/1815] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:06.981409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 911/1816] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:21:06.983180 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9ca93-5fd0-4706-a166-0156c9b00255 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:06.983625 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9ca93-5fd0-4706-a166-0156c9b00255 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:06.984050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5843262e-70b5-4295-a61e-90417c943c0a None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:06.984488 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9ca93-5fd0-4706-a166-0156c9b00255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:06.996635 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5843262e-70b5-4295-a61e-90417c943c0a None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:07.006106 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6767c33a-e74f-4779-a0ca-196a4da86e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:07.007608 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767c33a-e74f-4779-a0ca-196a4da86e03 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:07.007608 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767c33a-e74f-4779-a0ca-196a4da86e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:07.007819 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767c33a-e74f-4779-a0ca-196a4da86e03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:07.008669 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6767c33a-e74f-4779-a0ca-196a4da86e03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:21:07.008974 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6767c33a-e74f-4779-a0ca-196a4da86e03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:07.011077 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6767c33a-e74f-4779-a0ca-196a4da86e03 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:07.013226 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-22b9ca93-5fd0-4706-a166-0156c9b00255 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:07.014541 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-22b9ca93-5fd0-4706-a166-0156c9b00255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:07.028451 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 906/1818] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:21:07.028627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 912/1818] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:21:07.029491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6767c33a-e74f-4779-a0ca-196a4da86e03 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:07.029627 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b9ca93-5fd0-4706-a166-0156c9b00255 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:07.040149 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6767c33a-e74f-4779-a0ca-196a4da86e03 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:07.041685 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b9ca93-5fd0-4706-a166-0156c9b00255 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:07.047684 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5deae8a-d3ce-4247-a6c0-298813646216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:07.048309 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5deae8a-d3ce-4247-a6c0-298813646216 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:07.048595 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5deae8a-d3ce-4247-a6c0-298813646216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:07.048866 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5deae8a-d3ce-4247-a6c0-298813646216 admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:07.049550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5deae8a-d3ce-4247-a6c0-298813646216 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:21:07.049851 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5deae8a-d3ce-4247-a6c0-298813646216 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:07.051565 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5deae8a-d3ce-4247-a6c0-298813646216 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:07.051818 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d0fd2a-fb9d-449f-bf01-99e1ce87f3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:07.052446 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0fd2a-fb9d-449f-bf01-99e1ce87f3b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:07.052861 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0fd2a-fb9d-449f-bf01-99e1ce87f3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:07.053260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0fd2a-fb9d-449f-bf01-99e1ce87f3b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:07.054102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d0fd2a-fb9d-449f-bf01-99e1ce87f3b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:07.058214 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d0fd2a-fb9d-449f-bf01-99e1ce87f3b6 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:07.064783 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 907/1819] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:21:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:21:07.070857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5deae8a-d3ce-4247-a6c0-298813646216 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:07.083114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5deae8a-d3ce-4247-a6c0-298813646216 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:07.093310 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a0ab4-5a26-4d85-b46f-af059ad9a6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:07.095161 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a0ab4-5a26-4d85-b46f-af059ad9a6b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:07.095161 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a0ab4-5a26-4d85-b46f-af059ad9a6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:07.095652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a0ab4-5a26-4d85-b46f-af059ad9a6b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:07.096242 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a0ab4-5a26-4d85-b46f-af059ad9a6b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:07.103164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 913/1820] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:21:07 2022] POST /identity/v3/projects => generated 403 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:07.103164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d0fd2a-fb9d-449f-bf01-99e1ce87f3b6 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:07.104978 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-c6d0fd2a-fb9d-449f-bf01-99e1ce87f3b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:07.107552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a0ab4-5a26-4d85-b46f-af059ad9a6b1 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:07.151079 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 908/1821] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:21:07 2022] POST /identity/v3/projects => generated 381 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:07.151079 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a0ab4-5a26-4d85-b46f-af059ad9a6b1 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:07.199029 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d0fd2a-fb9d-449f-bf01-99e1ce87f3b6 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:07.220207 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a0ab4-5a26-4d85-b46f-af059ad9a6b1 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:07.225416 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7a5c4-4ee4-4b62-a500-bd5824377117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:07.226222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7a5c4-4ee4-4b62-a500-bd5824377117 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:07.226637 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7a5c4-4ee4-4b62-a500-bd5824377117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:07.226915 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7a5c4-4ee4-4b62-a500-bd5824377117 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:07.227583 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f7a5c4-4ee4-4b62-a500-bd5824377117 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:07.229521 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f7a5c4-4ee4-4b62-a500-bd5824377117 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:07.253674 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 909/1822] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:21:07 2022] POST /identity/v3/users => generated 302 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:07.253674 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7a5c4-4ee4-4b62-a500-bd5824377117 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:07.262284 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7a5c4-4ee4-4b62-a500-bd5824377117 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:07.264981 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f2cfd-50d7-4e7d-a06b-93856a7085f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:07.265782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f2cfd-50d7-4e7d-a06b-93856a7085f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:07.266071 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f2cfd-50d7-4e7d-a06b-93856a7085f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:07.266474 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f2cfd-50d7-4e7d-a06b-93856a7085f7 admin admin] PATH_INFO: `/v3/projects/891c3e6e45d945e59f6b41adfb62b381` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:07.267632 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2f2cfd-50d7-4e7d-a06b-93856a7085f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=891c3e6e45d945e59f6b41adfb62b381)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:07.268783 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff7bcf-46b2-4f49-a139-c81ecd0cf6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:07.269439 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff7bcf-46b2-4f49-a139-c81ecd0cf6f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:07.269619 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2f2cfd-50d7-4e7d-a06b-93856a7085f7 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:07.269726 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff7bcf-46b2-4f49-a139-c81ecd0cf6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:07.269984 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff7bcf-46b2-4f49-a139-c81ecd0cf6f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:07.270619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff7bcf-46b2-4f49-a139-c81ecd0cf6f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:07.273762 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff7bcf-46b2-4f49-a139-c81ecd0cf6f2 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:07.317902 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 910/1823] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:21:07 2022] POST /identity/v3/users => generated 291 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:07.317902 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff7bcf-46b2-4f49-a139-c81ecd0cf6f2 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:07.332619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff7bcf-46b2-4f49-a139-c81ecd0cf6f2 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:07.339384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:07.360803 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:07.368840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-ad2f2cfd-50d7-4e7d-a06b-93856a7085f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '891c3e6e45d945e59f6b41adfb62b381', 'request_id': 'req-ad2f2cfd-50d7-4e7d-a06b-93856a7085f7'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:07.379021 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-ad2f2cfd-50d7-4e7d-a06b-93856a7085f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '891c3e6e45d945e59f6b41adfb62b381', 'request_id': 'req-ad2f2cfd-50d7-4e7d-a06b-93856a7085f7'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:07.390560 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-ad2f2cfd-50d7-4e7d-a06b-93856a7085f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '891c3e6e45d945e59f6b41adfb62b381', 'request_id': 'req-ad2f2cfd-50d7-4e7d-a06b-93856a7085f7'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:07.419669 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd3a8c-1cee-4f19-98ea-bd63af420e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:07.420694 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd3a8c-1cee-4f19-98ea-bd63af420e7d admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:07.421166 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd3a8c-1cee-4f19-98ea-bd63af420e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:07.421337 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-ad2f2cfd-50d7-4e7d-a06b-93856a7085f7 admin admin] The token cache is being invalidate because project 891c3e6e45d945e59f6b41adfb62b381 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:21:07.421838 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-ad2f2cfd-50d7-4e7d-a06b-93856a7085f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:07.422426 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd3a8c-1cee-4f19-98ea-bd63af420e7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:07.422526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-ad2f2cfd-50d7-4e7d-a06b-93856a7085f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:07.423458 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd3a8c-1cee-4f19-98ea-bd63af420e7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:07.424356 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 914/1824] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:21:07 2022] DELETE /identity/v3/projects/891c3e6e45d945e59f6b41adfb62b381 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:07.425210 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f2cfd-50d7-4e7d-a06b-93856a7085f7 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:07.425452 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd3a8c-1cee-4f19-98ea-bd63af420e7d admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:07.428586 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-ad2f2cfd-50d7-4e7d-a06b-93856a7085f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:07.444250 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f2cfd-50d7-4e7d-a06b-93856a7085f7 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:07.450436 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49abd4-3a28-4744-be90-2ed04f897b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:07.451236 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49abd4-3a28-4744-be90-2ed04f897b76 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:07.451519 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49abd4-3a28-4744-be90-2ed04f897b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:07.451905 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49abd4-3a28-4744-be90-2ed04f897b76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:07.452518 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 911/1825] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:07.453384 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd3a8c-1cee-4f19-98ea-bd63af420e7d admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:07.455502 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49abd4-3a28-4744-be90-2ed04f897b76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:07.457423 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49abd4-3a28-4744-be90-2ed04f897b76 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:07.458916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-cfdd3a8c-1cee-4f19-98ea-bd63af420e7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:07.475084 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 915/1826] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:07.476321 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49abd4-3a28-4744-be90-2ed04f897b76 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:07.478354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-ef49abd4-3a28-4744-be90-2ed04f897b76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:07.495793 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd3a8c-1cee-4f19-98ea-bd63af420e7d admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:07.499209 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49abd4-3a28-4744-be90-2ed04f897b76 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:07.501168 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af0551-928b-4782-b202-6feeb325c340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:07.501815 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af0551-928b-4782-b202-6feeb325c340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:07.502099 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af0551-928b-4782-b202-6feeb325c340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:07.502378 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af0551-928b-4782-b202-6feeb325c340 admin admin] PATH_INFO: `/v3/users/7e0eba84e85f4acfa9c828c8aa76538d` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:07.503577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08af0551-928b-4782-b202-6feeb325c340 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e0eba84e85f4acfa9c828c8aa76538d)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:07.506286 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08af0551-928b-4782-b202-6feeb325c340 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:07.512431 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fcadc7-78df-4009-bb53-dbe915146d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:07.513556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fcadc7-78df-4009-bb53-dbe915146d3c admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:07.514231 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fcadc7-78df-4009-bb53-dbe915146d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:07.514704 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fcadc7-78df-4009-bb53-dbe915146d3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:07.515542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fcadc7-78df-4009-bb53-dbe915146d3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:07.517742 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fcadc7-78df-4009-bb53-dbe915146d3c admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:07.531374 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 916/1827] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:07.532489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fcadc7-78df-4009-bb53-dbe915146d3c admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:07.548984 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fcadc7-78df-4009-bb53-dbe915146d3c admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:07.559240 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0581ab-c5d5-4418-8cbc-deccd0861ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:07.560404 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0581ab-c5d5-4418-8cbc-deccd0861ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:07.561251 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0581ab-c5d5-4418-8cbc-deccd0861ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:07.561540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0581ab-c5d5-4418-8cbc-deccd0861ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:07.563481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0581ab-c5d5-4418-8cbc-deccd0861ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:07.566156 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0581ab-c5d5-4418-8cbc-deccd0861ba4 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:07.582771 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-08af0551-928b-4782-b202-6feeb325c340 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e0eba84e85f4acfa9c828c8aa76538d'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:07.583570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 917/1828] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:07.584514 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0581ab-c5d5-4418-8cbc-deccd0861ba4 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:07.595837 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0581ab-c5d5-4418-8cbc-deccd0861ba4 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:07.600987 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73999a10-9f7d-4f28-940d-a6f4400f972d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:07.601668 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73999a10-9f7d-4f28-940d-a6f4400f972d admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:07.601952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73999a10-9f7d-4f28-940d-a6f4400f972d admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:07.602231 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73999a10-9f7d-4f28-940d-a6f4400f972d admin admin] PATH_INFO: `/v3/projects/b09d25d389f14922b2313ba19cb5b569/users/c617ead6d8884fbb8499b2db118fadbc/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:07.621783 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73999a10-9f7d-4f28-940d-a6f4400f972d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b09d25d389f14922b2313ba19cb5b569, user_id=c617ead6d8884fbb8499b2db118fadbc, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:07.622385 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-08af0551-928b-4782-b202-6feeb325c340 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e0eba84e85f4acfa9c828c8aa76538d'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:07.624500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73999a10-9f7d-4f28-940d-a6f4400f972d admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:07.637399 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-08af0551-928b-4782-b202-6feeb325c340 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e0eba84e85f4acfa9c828c8aa76538d'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:07.643255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 918/1829] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:07 2022] PUT /identity/v3/projects/b09d25d389f14922b2313ba19cb5b569/users/c617ead6d8884fbb8499b2db118fadbc/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:07.644186 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73999a10-9f7d-4f28-940d-a6f4400f972d admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:07.650730 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-08af0551-928b-4782-b202-6feeb325c340 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e0eba84e85f4acfa9c828c8aa76538d'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:07.651622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-08af0551-928b-4782-b202-6feeb325c340 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e0eba84e85f4acfa9c828c8aa76538d'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:07.658805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 912/1830] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:21:07 2022] DELETE /identity/v3/users/7e0eba84e85f4acfa9c828c8aa76538d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:07.660194 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0b214-12ed-4411-beaa-2a7ca73e4ef9 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:07.660539 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0b214-12ed-4411-beaa-2a7ca73e4ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:07.660822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0b214-12ed-4411-beaa-2a7ca73e4ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:07.728308 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-7ca0b214-12ed-4411-beaa-2a7ca73e4ef9 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:07.738447 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-7ca0b214-12ed-4411-beaa-2a7ca73e4ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:07.755476 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73999a10-9f7d-4f28-940d-a6f4400f972d admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:07.867123 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 913/1831] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:07.873541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe2607-04ef-4582-aac2-ee4300398bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:07.874277 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe2607-04ef-4582-aac2-ee4300398bf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:07.874557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe2607-04ef-4582-aac2-ee4300398bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:07.874828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe2607-04ef-4582-aac2-ee4300398bf8 admin admin] PATH_INFO: `/v3/projects/819cac3ed48b4aea957ebdc161f194c7/users/dacd6c350a6e4e07b29cf27775c62fa5/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:07.876529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0b214-12ed-4411-beaa-2a7ca73e4ef9 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:07.878183 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-7ca0b214-12ed-4411-beaa-2a7ca73e4ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:07.899482 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe2607-04ef-4582-aac2-ee4300398bf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=819cac3ed48b4aea957ebdc161f194c7, user_id=dacd6c350a6e4e07b29cf27775c62fa5, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:07.903882 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe2607-04ef-4582-aac2-ee4300398bf8 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:07.917575 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:07.922553 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:07.935311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 919/1832] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:07 2022] PUT /identity/v3/projects/819cac3ed48b4aea957ebdc161f194c7/users/dacd6c350a6e4e07b29cf27775c62fa5/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:07.944087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36da182-cf85-409b-9dd5-c6560ef4c097 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:07.945130 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36da182-cf85-409b-9dd5-c6560ef4c097 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:07.945784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36da182-cf85-409b-9dd5-c6560ef4c097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:07.965100 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0b214-12ed-4411-beaa-2a7ca73e4ef9 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:07.984033 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-f36da182-cf85-409b-9dd5-c6560ef4c097 None None] MFA Rules not processed for user `dacd6c350a6e4e07b29cf27775c62fa5`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:07.985478 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f36da182-cf85-409b-9dd5-c6560ef4c097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:08.031224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f4e6d-630a-4315-9cc0-976db21c7774 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:08.031854 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f4e6d-630a-4315-9cc0-976db21c7774 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:08.032189 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f4e6d-630a-4315-9cc0-976db21c7774 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:08.032432 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f4e6d-630a-4315-9cc0-976db21c7774 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:08.074644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 920/1833] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:21:08.075720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36da182-cf85-409b-9dd5-c6560ef4c097 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:08.078873 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f36da182-cf85-409b-9dd5-c6560ef4c097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:08.105698 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f4e6d-630a-4315-9cc0-976db21c7774 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:08.107219 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f4e6d-630a-4315-9cc0-976db21c7774 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:08.117637 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 914/1834] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:21:08.119214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f4e6d-630a-4315-9cc0-976db21c7774 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:08.120601 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-e64f4e6d-630a-4315-9cc0-976db21c7774 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:08.136062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f4e6d-630a-4315-9cc0-976db21c7774 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:08.142372 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396a555-343c-4cda-9a60-5bddc79ac8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:08.142884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396a555-343c-4cda-9a60-5bddc79ac8d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:08.143210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396a555-343c-4cda-9a60-5bddc79ac8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:08.143549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396a555-343c-4cda-9a60-5bddc79ac8d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:08.144142 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396a555-343c-4cda-9a60-5bddc79ac8d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:08.146161 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396a555-343c-4cda-9a60-5bddc79ac8d7 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:08.161509 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36da182-cf85-409b-9dd5-c6560ef4c097 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:08.168156 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08206dd-16fc-4d91-8fa7-f1dd5582e47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:08.168819 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08206dd-16fc-4d91-8fa7-f1dd5582e47c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:08.169105 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08206dd-16fc-4d91-8fa7-f1dd5582e47c admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:08.169377 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08206dd-16fc-4d91-8fa7-f1dd5582e47c admin admin] PATH_INFO: `/v3/projects/c8d19f9281f54813b356c9e10ee6924b` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:08.171978 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08206dd-16fc-4d91-8fa7-f1dd5582e47c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8d19f9281f54813b356c9e10ee6924b)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:08.175172 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08206dd-16fc-4d91-8fa7-f1dd5582e47c admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:08.201054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 915/1835] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:21:08 2022] POST /identity/v3/projects => generated 383 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:08.202587 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396a555-343c-4cda-9a60-5bddc79ac8d7 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:08.246905 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-f08206dd-16fc-4d91-8fa7-f1dd5582e47c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8d19f9281f54813b356c9e10ee6924b', 'request_id': 'req-f08206dd-16fc-4d91-8fa7-f1dd5582e47c'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:08.253332 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-f08206dd-16fc-4d91-8fa7-f1dd5582e47c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8d19f9281f54813b356c9e10ee6924b', 'request_id': 'req-f08206dd-16fc-4d91-8fa7-f1dd5582e47c'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:08.262485 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-f08206dd-16fc-4d91-8fa7-f1dd5582e47c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8d19f9281f54813b356c9e10ee6924b', 'request_id': 'req-f08206dd-16fc-4d91-8fa7-f1dd5582e47c'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:08.277585 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396a555-343c-4cda-9a60-5bddc79ac8d7 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:08.278109 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-f08206dd-16fc-4d91-8fa7-f1dd5582e47c admin admin] The token cache is being invalidate because project c8d19f9281f54813b356c9e10ee6924b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:21:08.278109 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-f08206dd-16fc-4d91-8fa7-f1dd5582e47c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:08.278846 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-f08206dd-16fc-4d91-8fa7-f1dd5582e47c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:08.280801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 921/1836] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:21:08 2022] DELETE /identity/v3/projects/c8d19f9281f54813b356c9e10ee6924b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:08.285678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03485122-624d-461a-92c1-79e4e42198cf None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:08.286079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03485122-624d-461a-92c1-79e4e42198cf None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:08.286759 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03485122-624d-461a-92c1-79e4e42198cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:08.301808 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:08.310405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-03485122-624d-461a-92c1-79e4e42198cf None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:08.311763 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-03485122-624d-461a-92c1-79e4e42198cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:08.328857 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:08.356554 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac51399-9616-40d4-9d07-2489dfd243b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:08.357186 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac51399-9616-40d4-9d07-2489dfd243b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:08.357570 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac51399-9616-40d4-9d07-2489dfd243b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:08.357849 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac51399-9616-40d4-9d07-2489dfd243b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:08.358492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac51399-9616-40d4-9d07-2489dfd243b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:08.361220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac51399-9616-40d4-9d07-2489dfd243b6 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:08.386454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 916/1837] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:21:08 2022] POST /identity/v3/users => generated 298 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:21:08.392362 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac51399-9616-40d4-9d07-2489dfd243b6 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:08.395871 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-0ac51399-9616-40d4-9d07-2489dfd243b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:08.407420 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 922/1838] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:08.409231 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91d00b-9d69-4ebd-a952-498f7e915dd8 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:08.409524 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91d00b-9d69-4ebd-a952-498f7e915dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:08.409805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91d00b-9d69-4ebd-a952-498f7e915dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:08.427705 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac51399-9616-40d4-9d07-2489dfd243b6 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:08.442101 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998537cc-75f4-4d9a-bc89-cfd99d9bc8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:08.442792 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998537cc-75f4-4d9a-bc89-cfd99d9bc8a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:08.443095 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998537cc-75f4-4d9a-bc89-cfd99d9bc8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:08.443379 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998537cc-75f4-4d9a-bc89-cfd99d9bc8a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:08.444139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998537cc-75f4-4d9a-bc89-cfd99d9bc8a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:08.447966 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998537cc-75f4-4d9a-bc89-cfd99d9bc8a2 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:08.450255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-0c91d00b-9d69-4ebd-a952-498f7e915dd8 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:08.453390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-0c91d00b-9d69-4ebd-a952-498f7e915dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:08.480976 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 923/1839] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:08.483224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 917/1840] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:08.485818 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91d00b-9d69-4ebd-a952-498f7e915dd8 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:08.486980 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998537cc-75f4-4d9a-bc89-cfd99d9bc8a2 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:08.497910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91d00b-9d69-4ebd-a952-498f7e915dd8 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:08.504611 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5615d9c-85e0-4c49-b588-eaa7932773af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:08.505297 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5615d9c-85e0-4c49-b588-eaa7932773af admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:08.505631 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5615d9c-85e0-4c49-b588-eaa7932773af admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:08.505901 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5615d9c-85e0-4c49-b588-eaa7932773af admin admin] PATH_INFO: `/v3/projects/f27b008cf897454dab93b7923698c0ae/users/304e62e9abf245f382fcad96719423ed/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:08.515712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998537cc-75f4-4d9a-bc89-cfd99d9bc8a2 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:08.536585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5615d9c-85e0-4c49-b588-eaa7932773af admin admin] RBAC: Authorizing `identity:create_grant(project_id=f27b008cf897454dab93b7923698c0ae, user_id=304e62e9abf245f382fcad96719423ed, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:08.539216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039df8b4-48b4-4f35-a143-193ec765abc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:08.539872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039df8b4-48b4-4f35-a143-193ec765abc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:08.540197 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039df8b4-48b4-4f35-a143-193ec765abc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:08.540422 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039df8b4-48b4-4f35-a143-193ec765abc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:08.541134 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039df8b4-48b4-4f35-a143-193ec765abc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:21:08.541490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039df8b4-48b4-4f35-a143-193ec765abc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:08.542471 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5615d9c-85e0-4c49-b588-eaa7932773af admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:08.543190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039df8b4-48b4-4f35-a143-193ec765abc6 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:08.559602 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 918/1841] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:21:08.564787 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039df8b4-48b4-4f35-a143-193ec765abc6 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:08.579392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039df8b4-48b4-4f35-a143-193ec765abc6 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:08.588783 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 924/1842] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:08 2022] PUT /identity/v3/projects/f27b008cf897454dab93b7923698c0ae/users/304e62e9abf245f382fcad96719423ed/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:08.589978 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5615d9c-85e0-4c49-b588-eaa7932773af admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:08.657795 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f89688-c69d-479c-a1a2-958dcdfb1a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:08.658336 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f89688-c69d-479c-a1a2-958dcdfb1a7b admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:08.658634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f89688-c69d-479c-a1a2-958dcdfb1a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:08.661403 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f89688-c69d-479c-a1a2-958dcdfb1a7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:08.662202 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f89688-c69d-479c-a1a2-958dcdfb1a7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:08.666517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f89688-c69d-479c-a1a2-958dcdfb1a7b admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:08.738450 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 919/1843] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:21:08 2022] POST /identity/v3/projects => generated 367 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:21:08.743154 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f89688-c69d-479c-a1a2-958dcdfb1a7b admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:08.807583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5615d9c-85e0-4c49-b588-eaa7932773af admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:08.883446 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f89688-c69d-479c-a1a2-958dcdfb1a7b admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:08.900089 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b143b0-60f2-45f9-b39f-93a6e13e4451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:08.904561 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b143b0-60f2-45f9-b39f-93a6e13e4451 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:08.905934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b143b0-60f2-45f9-b39f-93a6e13e4451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:08.907026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b143b0-60f2-45f9-b39f-93a6e13e4451 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:08.911154 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b143b0-60f2-45f9-b39f-93a6e13e4451 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:08.917212 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b143b0-60f2-45f9-b39f-93a6e13e4451 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:08.998342 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 920/1844] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:21:08 2022] POST /identity/v3/users => generated 284 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:21:09.000539 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b143b0-60f2-45f9-b39f-93a6e13e4451 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:09.009127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:09.020533 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebf55d-b52d-497d-8da0-6a4633004de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:09.020852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebf55d-b52d-497d-8da0-6a4633004de6 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:09.021133 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebf55d-b52d-497d-8da0-6a4633004de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:09.021409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebf55d-b52d-497d-8da0-6a4633004de6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:09.022183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbebf55d-b52d-497d-8da0-6a4633004de6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:09.024325 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbebf55d-b52d-497d-8da0-6a4633004de6 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:09.034093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b143b0-60f2-45f9-b39f-93a6e13e4451 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:09.046332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d51f65-9ec4-4261-a19d-61415d894f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:09.047223 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 925/1845] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:09.047526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d51f65-9ec4-4261-a19d-61415d894f7a admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:09.048066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d51f65-9ec4-4261-a19d-61415d894f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:09.048359 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d51f65-9ec4-4261-a19d-61415d894f7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:09.049025 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d51f65-9ec4-4261-a19d-61415d894f7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:09.050112 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebf55d-b52d-497d-8da0-6a4633004de6 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:09.066180 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d51f65-9ec4-4261-a19d-61415d894f7a admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:09.092299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebf55d-b52d-497d-8da0-6a4633004de6 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:09.093401 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 921/1846] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:09.095428 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d51f65-9ec4-4261-a19d-61415d894f7a admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:09.106954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d51f65-9ec4-4261-a19d-61415d894f7a admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:09.113995 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb138395-3fbf-49f2-83e9-b5d75d277105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:09.114973 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb138395-3fbf-49f2-83e9-b5d75d277105 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:09.115274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb138395-3fbf-49f2-83e9-b5d75d277105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:09.115720 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb138395-3fbf-49f2-83e9-b5d75d277105 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:09.117255 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb138395-3fbf-49f2-83e9-b5d75d277105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:09.120503 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb138395-3fbf-49f2-83e9-b5d75d277105 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:09.122191 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51b348-6e74-4864-97d1-75dd3f10c0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:09.124297 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51b348-6e74-4864-97d1-75dd3f10c0ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:09.128558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51b348-6e74-4864-97d1-75dd3f10c0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:09.129164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51b348-6e74-4864-97d1-75dd3f10c0ee admin admin] PATH_INFO: `/v3/projects/f27b008cf897454dab93b7923698c0ae/users/304e62e9abf245f382fcad96719423ed/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:09.131868 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51b348-6e74-4864-97d1-75dd3f10c0ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=f27b008cf897454dab93b7923698c0ae, user_id=304e62e9abf245f382fcad96719423ed, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:09.137977 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51b348-6e74-4864-97d1-75dd3f10c0ee admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:09.141565 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 922/1847] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:09.142545 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb138395-3fbf-49f2-83e9-b5d75d277105 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:09.158492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb138395-3fbf-49f2-83e9-b5d75d277105 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:09.164778 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 926/1848] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:09 2022] PUT /identity/v3/projects/f27b008cf897454dab93b7923698c0ae/users/304e62e9abf245f382fcad96719423ed/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:21:09.176135 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1dfd5c-1288-47ff-a653-7e664583c120 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:09.176777 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1dfd5c-1288-47ff-a653-7e664583c120 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:09.177375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1dfd5c-1288-47ff-a653-7e664583c120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:09.220079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-3d1dfd5c-1288-47ff-a653-7e664583c120 None None] MFA Rules not processed for user `304e62e9abf245f382fcad96719423ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:09.221719 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-3d1dfd5c-1288-47ff-a653-7e664583c120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:09.230717 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:09.288176 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06dfc8-c193-4b57-b948-22cc5e56639c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:09.289703 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06dfc8-c193-4b57-b948-22cc5e56639c admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:09.290038 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06dfc8-c193-4b57-b948-22cc5e56639c admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:09.290322 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06dfc8-c193-4b57-b948-22cc5e56639c admin admin] PATH_INFO: `/v3/projects/d51b981cd6a6411c9d4c0fecd834db22/users/4f362d1d375747cda66f8b7bbdc2bd95/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:09.310168 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06dfc8-c193-4b57-b948-22cc5e56639c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d51b981cd6a6411c9d4c0fecd834db22, user_id=4f362d1d375747cda66f8b7bbdc2bd95, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:09.313333 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06dfc8-c193-4b57-b948-22cc5e56639c admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:09.332613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 923/1849] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:09 2022] PUT /identity/v3/projects/d51b981cd6a6411c9d4c0fecd834db22/users/4f362d1d375747cda66f8b7bbdc2bd95/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:09.334144 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 927/1850] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:09 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:21:09.352608 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1dfd5c-1288-47ff-a653-7e664583c120 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:09.353655 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa527be-f323-4ae0-a470-5aa3ee587244 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:09.353955 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa527be-f323-4ae0-a470-5aa3ee587244 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:09.354145 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-3d1dfd5c-1288-47ff-a653-7e664583c120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:09.354255 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa527be-f323-4ae0-a470-5aa3ee587244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:09.398052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-5fa527be-f323-4ae0-a470-5aa3ee587244 None None] MFA Rules not processed for user `4f362d1d375747cda66f8b7bbdc2bd95`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:09.399969 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-5fa527be-f323-4ae0-a470-5aa3ee587244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:09.438713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1dfd5c-1288-47ff-a653-7e664583c120 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:09.494185 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 924/1851] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:09.498445 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa527be-f323-4ae0-a470-5aa3ee587244 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:09.510776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e50cb-cdd4-4832-9064-6b7d363760eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:09.511258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e50cb-cdd4-4832-9064-6b7d363760eb service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:09.511556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e50cb-cdd4-4832-9064-6b7d363760eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:09.512080 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e50cb-cdd4-4832-9064-6b7d363760eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:09.605706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:09.617690 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e50cb-cdd4-4832-9064-6b7d363760eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:09.618678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e50cb-cdd4-4832-9064-6b7d363760eb service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:09.630487 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 928/1852] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:09 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:21:09.632982 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa527be-f323-4ae0-a470-5aa3ee587244 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:09.641036 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a6c98-1fb8-4bb8-8d46-5dc67bc353b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:09.641868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a6c98-1fb8-4bb8-8d46-5dc67bc353b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:09.642193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a6c98-1fb8-4bb8-8d46-5dc67bc353b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:09.642792 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a6c98-1fb8-4bb8-8d46-5dc67bc353b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:09.643937 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a6c98-1fb8-4bb8-8d46-5dc67bc353b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:09.646210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a6c98-1fb8-4bb8-8d46-5dc67bc353b3 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:09.689858 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 925/1853] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:21:09 2022] POST /identity/v3/projects => generated 365 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:09.691752 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e50cb-cdd4-4832-9064-6b7d363760eb service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:09.734319 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e50cb-cdd4-4832-9064-6b7d363760eb service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:09.746793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:09.767992 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a6c98-1fb8-4bb8-8d46-5dc67bc353b3 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:09.823892 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1679cba-10bb-4dc7-8da8-576863c12441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:09.825334 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1679cba-10bb-4dc7-8da8-576863c12441 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:09.825761 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1679cba-10bb-4dc7-8da8-576863c12441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:09.826321 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1679cba-10bb-4dc7-8da8-576863c12441 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:09.827076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1679cba-10bb-4dc7-8da8-576863c12441 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:09.829133 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1679cba-10bb-4dc7-8da8-576863c12441 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:09.853494 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a6c98-1fb8-4bb8-8d46-5dc67bc353b3 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:09.858143 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540b82c-bcbe-4ffe-b023-b72954b48605 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:09.859028 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540b82c-bcbe-4ffe-b023-b72954b48605 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:09.859486 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540b82c-bcbe-4ffe-b023-b72954b48605 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:09.859945 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540b82c-bcbe-4ffe-b023-b72954b48605 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:09.860370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 929/1854] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:21:09 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:09.862149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-7540b82c-bcbe-4ffe-b023-b72954b48605 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:09.864032 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1679cba-10bb-4dc7-8da8-576863c12441 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:09.878171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1679cba-10bb-4dc7-8da8-576863c12441 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:09.885951 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2e547-ed0a-4463-8e45-9fcf2b49912c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:09.886679 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2e547-ed0a-4463-8e45-9fcf2b49912c admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:09.887112 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2e547-ed0a-4463-8e45-9fcf2b49912c admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:09.887501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2e547-ed0a-4463-8e45-9fcf2b49912c admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:09.888235 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d2e547-ed0a-4463-8e45-9fcf2b49912c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:09.890503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d2e547-ed0a-4463-8e45-9fcf2b49912c admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:09.905997 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 930/1855] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:09.909978 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2e547-ed0a-4463-8e45-9fcf2b49912c admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:09.927551 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2e547-ed0a-4463-8e45-9fcf2b49912c admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:09.935294 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686f788-0175-44a7-b8d6-78feea79e599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:09.936102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686f788-0175-44a7-b8d6-78feea79e599 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:09.936437 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686f788-0175-44a7-b8d6-78feea79e599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:09.936815 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686f788-0175-44a7-b8d6-78feea79e599 admin admin] PATH_INFO: `/v3/projects/2b571aadc71c4a36822a99f995203e63/users/31dfbeae8ea84958aee5a69280449c6f/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:09.951309 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7540b82c-bcbe-4ffe-b023-b72954b48605 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:09.953611 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7540b82c-bcbe-4ffe-b023-b72954b48605 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:09.956085 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686f788-0175-44a7-b8d6-78feea79e599 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b571aadc71c4a36822a99f995203e63, user_id=31dfbeae8ea84958aee5a69280449c6f, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:09.959120 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686f788-0175-44a7-b8d6-78feea79e599 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:09.981167 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 926/1856] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:21:10.003667 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 931/1857] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:09 2022] PUT /identity/v3/projects/2b571aadc71c4a36822a99f995203e63/users/31dfbeae8ea84958aee5a69280449c6f/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:21:10.008128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540b82c-bcbe-4ffe-b023-b72954b48605 service cinder] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:10.025078 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540b82c-bcbe-4ffe-b023-b72954b48605 service cinder] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:10.173673 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ffcac-35d4-4f70-b4a7-b6112e0226d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:10.174476 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ffcac-35d4-4f70-b4a7-b6112e0226d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:10.174695 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ffcac-35d4-4f70-b4a7-b6112e0226d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:10.175066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ffcac-35d4-4f70-b4a7-b6112e0226d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:10.177113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84ffcac-35d4-4f70-b4a7-b6112e0226d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:10.177289 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84ffcac-35d4-4f70-b4a7-b6112e0226d0 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:10.198851 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 927/1858] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:10 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:10.203133 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686f788-0175-44a7-b8d6-78feea79e599 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:10.228498 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686f788-0175-44a7-b8d6-78feea79e599 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:10.247078 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec54c3d-21f0-43ff-bfe7-81304f38e5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:10.247078 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec54c3d-21f0-43ff-bfe7-81304f38e5e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:10.247362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec54c3d-21f0-43ff-bfe7-81304f38e5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:10.247577 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec54c3d-21f0-43ff-bfe7-81304f38e5e9 admin admin] PATH_INFO: `/v3/projects/2b571aadc71c4a36822a99f995203e63/users/31dfbeae8ea84958aee5a69280449c6f/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:10.252292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec54c3d-21f0-43ff-bfe7-81304f38e5e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b571aadc71c4a36822a99f995203e63, user_id=31dfbeae8ea84958aee5a69280449c6f, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:10.255403 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec54c3d-21f0-43ff-bfe7-81304f38e5e9 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:10.269098 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 932/1859] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:10 2022] PUT /identity/v3/projects/2b571aadc71c4a36822a99f995203e63/users/31dfbeae8ea84958aee5a69280449c6f/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:10.282940 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e46be8-da27-4100-bdb5-4f28f181c032 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:10.283307 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e46be8-da27-4100-bdb5-4f28f181c032 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:10.283531 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e46be8-da27-4100-bdb5-4f28f181c032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:10.341105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-e4e46be8-da27-4100-bdb5-4f28f181c032 None None] MFA Rules not processed for user `31dfbeae8ea84958aee5a69280449c6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:10.342262 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-e4e46be8-da27-4100-bdb5-4f28f181c032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:10.390731 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:10.479732 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec54c3d-21f0-43ff-bfe7-81304f38e5e9 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:10.516941 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 928/1860] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:10 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:10.530032 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e46be8-da27-4100-bdb5-4f28f181c032 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:10.531986 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-e4e46be8-da27-4100-bdb5-4f28f181c032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:10.615362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec54c3d-21f0-43ff-bfe7-81304f38e5e9 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:10.621217 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c2830-1139-449f-9ccc-67659bd3cf5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:10.621964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c2830-1139-449f-9ccc-67659bd3cf5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:10.622437 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c2830-1139-449f-9ccc-67659bd3cf5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:10.622782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c2830-1139-449f-9ccc-67659bd3cf5b admin admin] PATH_INFO: `/v3/users/dacd6c350a6e4e07b29cf27775c62fa5` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:10.624444 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071c2830-1139-449f-9ccc-67659bd3cf5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dacd6c350a6e4e07b29cf27775c62fa5)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:10.626828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071c2830-1139-449f-9ccc-67659bd3cf5b admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:10.640966 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e46be8-da27-4100-bdb5-4f28f181c032 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:10.645947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c506f-ece9-48c2-a9da-191ec2e4e645 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:10.647634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c506f-ece9-48c2-a9da-191ec2e4e645 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:10.648572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c506f-ece9-48c2-a9da-191ec2e4e645 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:10.649226 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c506f-ece9-48c2-a9da-191ec2e4e645 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:10.662363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c506f-ece9-48c2-a9da-191ec2e4e645 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:10.664992 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c506f-ece9-48c2-a9da-191ec2e4e645 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:10.690530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 929/1861] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:10 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:21:10.694562 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-071c2830-1139-449f-9ccc-67659bd3cf5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dacd6c350a6e4e07b29cf27775c62fa5'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:10.700044 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:10.705829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-071c2830-1139-449f-9ccc-67659bd3cf5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dacd6c350a6e4e07b29cf27775c62fa5'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:10.719848 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-071c2830-1139-449f-9ccc-67659bd3cf5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dacd6c350a6e4e07b29cf27775c62fa5'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:10.721030 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-071c2830-1139-449f-9ccc-67659bd3cf5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dacd6c350a6e4e07b29cf27775c62fa5'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:10.722173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-071c2830-1139-449f-9ccc-67659bd3cf5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dacd6c350a6e4e07b29cf27775c62fa5'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:10.761750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 933/1862] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:21:10 2022] DELETE /identity/v3/users/dacd6c350a6e4e07b29cf27775c62fa5 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:10.767791 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4304de-5dfe-4c26-bea0-74d43d361582 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:10.768118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4304de-5dfe-4c26-bea0-74d43d361582 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:10.768516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4304de-5dfe-4c26-bea0-74d43d361582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:10.799519 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-5b4304de-5dfe-4c26-bea0-74d43d361582 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:10.801025 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-5b4304de-5dfe-4c26-bea0-74d43d361582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:10.801896 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c2830-1139-449f-9ccc-67659bd3cf5b admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:10.803669 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-071c2830-1139-449f-9ccc-67659bd3cf5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:10.904975 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c2830-1139-449f-9ccc-67659bd3cf5b admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:10.910108 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c2b9ad-079e-4d63-9358-3e4ca12b8c8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:10.911380 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2b9ad-079e-4d63-9358-3e4ca12b8c8a service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:10.912024 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2b9ad-079e-4d63-9358-3e4ca12b8c8a service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:10.912510 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2b9ad-079e-4d63-9358-3e4ca12b8c8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:10.916230 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-85c2b9ad-079e-4d63-9358-3e4ca12b8c8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:10.931102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 930/1863] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:10.967071 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4304de-5dfe-4c26-bea0-74d43d361582 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:10.969873 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-5b4304de-5dfe-4c26-bea0-74d43d361582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:11.078504 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c2b9ad-079e-4d63-9358-3e4ca12b8c8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:11.081685 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c2b9ad-079e-4d63-9358-3e4ca12b8c8a service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:11.103834 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 934/1864] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:21:11.140037 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4304de-5dfe-4c26-bea0-74d43d361582 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:11.150118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180f140-56d0-45bc-a421-28eefdc529f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:11.150527 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180f140-56d0-45bc-a421-28eefdc529f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:11.150812 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180f140-56d0-45bc-a421-28eefdc529f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:11.151108 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180f140-56d0-45bc-a421-28eefdc529f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:11.169908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2180f140-56d0-45bc-a421-28eefdc529f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:11.171761 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2180f140-56d0-45bc-a421-28eefdc529f9 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:11.178815 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:11.187656 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 931/1865] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:21:11.559570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ba4ae-fac9-42a4-9bcf-dbcd9f70944f None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:11.559894 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ba4ae-fac9-42a4-9bcf-dbcd9f70944f None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:11.560173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ba4ae-fac9-42a4-9bcf-dbcd9f70944f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:11.674817 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-f22ba4ae-fac9-42a4-9bcf-dbcd9f70944f None None] MFA Rules not processed for user `c617ead6d8884fbb8499b2db118fadbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:11.678364 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f22ba4ae-fac9-42a4-9bcf-dbcd9f70944f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:11.892039 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 935/1866] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:11 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:21:11.897549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180f140-56d0-45bc-a421-28eefdc529f9 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:11.900321 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-2180f140-56d0-45bc-a421-28eefdc529f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:11.929825 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180f140-56d0-45bc-a421-28eefdc529f9 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:11.947792 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c794d5-153d-47f7-b092-906c03dba3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:11.948449 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c794d5-153d-47f7-b092-906c03dba3d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:11.948732 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c794d5-153d-47f7-b092-906c03dba3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:11.949002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c794d5-153d-47f7-b092-906c03dba3d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:11.949662 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c794d5-153d-47f7-b092-906c03dba3d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:11.951657 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c794d5-153d-47f7-b092-906c03dba3d3 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:12.004605 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 932/1867] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:21:11 2022] POST /identity/v3/projects => generated 403 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:12.009589 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ba4ae-fac9-42a4-9bcf-dbcd9f70944f None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:12.023256 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ba4ae-fac9-42a4-9bcf-dbcd9f70944f None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:12.041800 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:12.137157 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c03c3b-e1b0-43bf-9424-157731a89e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:12.137847 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c03c3b-e1b0-43bf-9424-157731a89e57 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:12.138464 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c03c3b-e1b0-43bf-9424-157731a89e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:12.138871 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c03c3b-e1b0-43bf-9424-157731a89e57 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:12.139956 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c03c3b-e1b0-43bf-9424-157731a89e57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:12.143156 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c03c3b-e1b0-43bf-9424-157731a89e57 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:12.183292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 936/1868] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:21:12 2022] POST /identity/v3/users => generated 308 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:12.186988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c794d5-153d-47f7-b092-906c03dba3d3 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:12.206779 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c794d5-153d-47f7-b092-906c03dba3d3 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:12.215293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debbae03-5167-4b8b-8fad-e05bd9c38046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:12.216375 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbae03-5167-4b8b-8fad-e05bd9c38046 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:12.216813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbae03-5167-4b8b-8fad-e05bd9c38046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:12.217185 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbae03-5167-4b8b-8fad-e05bd9c38046 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:12.218032 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debbae03-5167-4b8b-8fad-e05bd9c38046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:12.221536 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debbae03-5167-4b8b-8fad-e05bd9c38046 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:12.243085 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 933/1869] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:12.246808 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c03c3b-e1b0-43bf-9424-157731a89e57 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:12.262903 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c03c3b-e1b0-43bf-9424-157731a89e57 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:12.272818 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929b5df-f0d4-4697-8a9d-56f00dec16a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:12.275139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929b5df-f0d4-4697-8a9d-56f00dec16a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:12.275733 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929b5df-f0d4-4697-8a9d-56f00dec16a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:12.276254 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929b5df-f0d4-4697-8a9d-56f00dec16a5 admin admin] PATH_INFO: `/v3/projects/cdc27c76fc0d4f25a85e82d8e1535828/users/8ff416487627432ab095c864e2da8853/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:12.310615 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1929b5df-f0d4-4697-8a9d-56f00dec16a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdc27c76fc0d4f25a85e82d8e1535828, user_id=8ff416487627432ab095c864e2da8853, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:12.313651 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1929b5df-f0d4-4697-8a9d-56f00dec16a5 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:12.334809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 937/1870] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:12 2022] PUT /identity/v3/projects/cdc27c76fc0d4f25a85e82d8e1535828/users/8ff416487627432ab095c864e2da8853/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:12.338706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debbae03-5167-4b8b-8fad-e05bd9c38046 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:12.356608 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debbae03-5167-4b8b-8fad-e05bd9c38046 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:12.449474 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482792f2-95f8-490d-acc5-2e7a281d52fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:12.450114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482792f2-95f8-490d-acc5-2e7a281d52fc admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:12.450392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482792f2-95f8-490d-acc5-2e7a281d52fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:12.450727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482792f2-95f8-490d-acc5-2e7a281d52fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:12.451443 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482792f2-95f8-490d-acc5-2e7a281d52fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:12.453284 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482792f2-95f8-490d-acc5-2e7a281d52fc admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:12.466352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 934/1871] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:12.470334 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929b5df-f0d4-4697-8a9d-56f00dec16a5 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:12.489801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929b5df-f0d4-4697-8a9d-56f00dec16a5 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:12.498259 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e878c7-4c1b-4175-9f45-be756ed14248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:12.500519 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e878c7-4c1b-4175-9f45-be756ed14248 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:12.501053 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e878c7-4c1b-4175-9f45-be756ed14248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:12.501596 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e878c7-4c1b-4175-9f45-be756ed14248 admin admin] PATH_INFO: `/v3/projects/cdc27c76fc0d4f25a85e82d8e1535828/users/8ff416487627432ab095c864e2da8853/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:12.504242 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e878c7-4c1b-4175-9f45-be756ed14248 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdc27c76fc0d4f25a85e82d8e1535828, user_id=8ff416487627432ab095c864e2da8853, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:12.507854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e878c7-4c1b-4175-9f45-be756ed14248 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:12.543117 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 938/1872] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:12 2022] PUT /identity/v3/projects/cdc27c76fc0d4f25a85e82d8e1535828/users/8ff416487627432ab095c864e2da8853/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:12.618529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f669f6a-45d3-4092-a2b9-72633a21c37c None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:12.618813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f669f6a-45d3-4092-a2b9-72633a21c37c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:12.619131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f669f6a-45d3-4092-a2b9-72633a21c37c None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:12.620662 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 935/1873] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 31 00:21:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:21:12.650125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5188a-6d72-4ccc-b67e-8aaddf6fe4f0 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:12.650426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5188a-6d72-4ccc-b67e-8aaddf6fe4f0 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:12.650680 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5188a-6d72-4ccc-b67e-8aaddf6fe4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:12.681117 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-89d5188a-6d72-4ccc-b67e-8aaddf6fe4f0 None None] MFA Rules not processed for user `3aa53ae5a30146f69d7d6eefe4290dba`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:12.682684 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-89d5188a-6d72-4ccc-b67e-8aaddf6fe4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:12.853470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f669f6a-45d3-4092-a2b9-72633a21c37c None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:12.858534 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-4f669f6a-45d3-4092-a2b9-72633a21c37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:12.959348 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 939/1874] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 31 00:21:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:12.973298 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d5188a-6d72-4ccc-b67e-8aaddf6fe4f0 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:12.978093 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-89d5188a-6d72-4ccc-b67e-8aaddf6fe4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:13.004421 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d5188a-6d72-4ccc-b67e-8aaddf6fe4f0 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:13.017651 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba94bb9-c054-4d07-914d-fee436e81cf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:13.019249 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba94bb9-c054-4d07-914d-fee436e81cf0 service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:13.019249 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba94bb9-c054-4d07-914d-fee436e81cf0 service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:13.019249 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba94bb9-c054-4d07-914d-fee436e81cf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:13.047234 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba94bb9-c054-4d07-914d-fee436e81cf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:13.050818 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba94bb9-c054-4d07-914d-fee436e81cf0 service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:13.077948 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f669f6a-45d3-4092-a2b9-72633a21c37c None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:13.078645 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 940/1875] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:21:13.085046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a2c71-df1e-4b09-8290-835010d3a926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:13.086804 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a2c71-df1e-4b09-8290-835010d3a926 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:13.087234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a2c71-df1e-4b09-8290-835010d3a926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:13.087590 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a2c71-df1e-4b09-8290-835010d3a926 admin admin] PATH_INFO: `/v3/projects/819cac3ed48b4aea957ebdc161f194c7` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:13.088984 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1a2c71-df1e-4b09-8290-835010d3a926 admin admin] RBAC: Authorizing `identity:delete_project(project_id=819cac3ed48b4aea957ebdc161f194c7)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:13.092491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1a2c71-df1e-4b09-8290-835010d3a926 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:13.185883 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-dc1a2c71-df1e-4b09-8290-835010d3a926 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '819cac3ed48b4aea957ebdc161f194c7', 'request_id': 'req-dc1a2c71-df1e-4b09-8290-835010d3a926'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:13.196661 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-dc1a2c71-df1e-4b09-8290-835010d3a926 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '819cac3ed48b4aea957ebdc161f194c7', 'request_id': 'req-dc1a2c71-df1e-4b09-8290-835010d3a926'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:13.207173 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-dc1a2c71-df1e-4b09-8290-835010d3a926 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '819cac3ed48b4aea957ebdc161f194c7', 'request_id': 'req-dc1a2c71-df1e-4b09-8290-835010d3a926'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:13.208404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:13.228439 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-dc1a2c71-df1e-4b09-8290-835010d3a926 admin admin] The token cache is being invalidate because project 819cac3ed48b4aea957ebdc161f194c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:21:13.228820 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-dc1a2c71-df1e-4b09-8290-835010d3a926 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:13.229562 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-dc1a2c71-df1e-4b09-8290-835010d3a926 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:13.231789 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 936/1876] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:21:12 2022] DELETE /identity/v3/projects/819cac3ed48b4aea957ebdc161f194c7 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:13.237009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba94bb9-c054-4d07-914d-fee436e81cf0 service nova] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:13.256291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba94bb9-c054-4d07-914d-fee436e81cf0 service nova] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:13.321598 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:13.345437 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a34e3-2b93-492d-ad75-bb42cede3a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:13.347011 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a34e3-2b93-492d-ad75-bb42cede3a04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:13.347426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a34e3-2b93-492d-ad75-bb42cede3a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:13.347777 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a34e3-2b93-492d-ad75-bb42cede3a04 admin admin] PATH_INFO: `/v3/users/304e62e9abf245f382fcad96719423ed` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:13.349253 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a34e3-2b93-492d-ad75-bb42cede3a04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=304e62e9abf245f382fcad96719423ed)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:13.353606 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a34e3-2b93-492d-ad75-bb42cede3a04 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:13.430563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-b10a34e3-2b93-492d-ad75-bb42cede3a04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '304e62e9abf245f382fcad96719423ed'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:13.442979 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-b10a34e3-2b93-492d-ad75-bb42cede3a04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '304e62e9abf245f382fcad96719423ed'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:13.455362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-b10a34e3-2b93-492d-ad75-bb42cede3a04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '304e62e9abf245f382fcad96719423ed'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:13.456391 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-b10a34e3-2b93-492d-ad75-bb42cede3a04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '304e62e9abf245f382fcad96719423ed'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:13.457467 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-b10a34e3-2b93-492d-ad75-bb42cede3a04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '304e62e9abf245f382fcad96719423ed'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:13.488388 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 941/1877] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:21:13 2022] DELETE /identity/v3/users/304e62e9abf245f382fcad96719423ed => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:13.847069 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a2c71-df1e-4b09-8290-835010d3a926 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:13.848522 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-dc1a2c71-df1e-4b09-8290-835010d3a926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:13.863919 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a2c71-df1e-4b09-8290-835010d3a926 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:13.936035 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3da25-2be3-45ad-92fa-5c5dc52bc7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:13.936681 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3da25-2be3-45ad-92fa-5c5dc52bc7eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:13.936960 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3da25-2be3-45ad-92fa-5c5dc52bc7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:13.937234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3da25-2be3-45ad-92fa-5c5dc52bc7eb admin admin] PATH_INFO: `/v3/projects/f27b008cf897454dab93b7923698c0ae` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:13.938363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d3da25-2be3-45ad-92fa-5c5dc52bc7eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f27b008cf897454dab93b7923698c0ae)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:13.940504 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d3da25-2be3-45ad-92fa-5c5dc52bc7eb admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:14.002087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35afbb-23e7-4c87-beb3-72bcc2a45c4c None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:14.002448 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35afbb-23e7-4c87-beb3-72bcc2a45c4c None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:14.002793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35afbb-23e7-4c87-beb3-72bcc2a45c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:14.015746 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b8d3da25-2be3-45ad-92fa-5c5dc52bc7eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f27b008cf897454dab93b7923698c0ae', 'request_id': 'req-b8d3da25-2be3-45ad-92fa-5c5dc52bc7eb'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:14.024599 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b8d3da25-2be3-45ad-92fa-5c5dc52bc7eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f27b008cf897454dab93b7923698c0ae', 'request_id': 'req-b8d3da25-2be3-45ad-92fa-5c5dc52bc7eb'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:14.036958 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b8d3da25-2be3-45ad-92fa-5c5dc52bc7eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f27b008cf897454dab93b7923698c0ae', 'request_id': 'req-b8d3da25-2be3-45ad-92fa-5c5dc52bc7eb'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:14.046472 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-bc35afbb-23e7-4c87-beb3-72bcc2a45c4c None None] MFA Rules not processed for user `8ff416487627432ab095c864e2da8853`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:14.047794 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-bc35afbb-23e7-4c87-beb3-72bcc2a45c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:14.052286 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b8d3da25-2be3-45ad-92fa-5c5dc52bc7eb admin admin] The token cache is being invalidate because project f27b008cf897454dab93b7923698c0ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:21:14.052660 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b8d3da25-2be3-45ad-92fa-5c5dc52bc7eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:14.053404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b8d3da25-2be3-45ad-92fa-5c5dc52bc7eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:14.055714 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 937/1878] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:21:13 2022] DELETE /identity/v3/projects/f27b008cf897454dab93b7923698c0ae => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:14.061354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b03c0-176f-4785-82f6-241c6a3d6fea None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:14.062350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b03c0-176f-4785-82f6-241c6a3d6fea None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:14.062705 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b03c0-176f-4785-82f6-241c6a3d6fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:14.090212 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-b81b03c0-176f-4785-82f6-241c6a3d6fea None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:14.091560 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b81b03c0-176f-4785-82f6-241c6a3d6fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:14.194326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 942/1879] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:14 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:14.204589 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 938/1880] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:14.213221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2134d-08da-4632-b2c8-570d7ec4fee7 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:14.213984 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2134d-08da-4632-b2c8-570d7ec4fee7 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:14.214458 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2134d-08da-4632-b2c8-570d7ec4fee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:14.261567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-eca2134d-08da-4632-b2c8-570d7ec4fee7 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:14.268351 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-eca2134d-08da-4632-b2c8-570d7ec4fee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:14.284739 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 943/1881] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:14.287102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b03c0-176f-4785-82f6-241c6a3d6fea None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:14.301252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b03c0-176f-4785-82f6-241c6a3d6fea None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:14.307603 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a6331-d5b7-4bfb-8342-18fe7329c856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:14.308240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a6331-d5b7-4bfb-8342-18fe7329c856 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:14.308524 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a6331-d5b7-4bfb-8342-18fe7329c856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:14.308798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a6331-d5b7-4bfb-8342-18fe7329c856 admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:14.309603 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a6331-d5b7-4bfb-8342-18fe7329c856 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:21:14.309919 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a6331-d5b7-4bfb-8342-18fe7329c856 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:14.312436 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a6331-d5b7-4bfb-8342-18fe7329c856 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:14.334063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 939/1882] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:21:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:21:14.335324 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2134d-08da-4632-b2c8-570d7ec4fee7 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:14.347228 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2134d-08da-4632-b2c8-570d7ec4fee7 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:14.354067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfa315-3dcc-426b-b285-2ef671a64940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:14.355625 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfa315-3dcc-426b-b285-2ef671a64940 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:14.356044 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfa315-3dcc-426b-b285-2ef671a64940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:14.357290 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfa315-3dcc-426b-b285-2ef671a64940 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:14.357290 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfa315-3dcc-426b-b285-2ef671a64940 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:14.360517 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfa315-3dcc-426b-b285-2ef671a64940 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:14.409347 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 944/1883] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:21:14 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:21:14.414122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a6331-d5b7-4bfb-8342-18fe7329c856 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:14.439785 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a6331-d5b7-4bfb-8342-18fe7329c856 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:14.458560 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:14.580782 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017f28f-7974-49f2-9309-869988fe21f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:14.581831 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017f28f-7974-49f2-9309-869988fe21f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:14.582143 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017f28f-7974-49f2-9309-869988fe21f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:14.583191 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017f28f-7974-49f2-9309-869988fe21f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:14.587132 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4017f28f-7974-49f2-9309-869988fe21f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:14.590329 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4017f28f-7974-49f2-9309-869988fe21f6 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:14.654430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 940/1884] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:21:14 2022] POST /identity/v3/users => generated 292 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:14.659168 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfa315-3dcc-426b-b285-2ef671a64940 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:14.672390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfa315-3dcc-426b-b285-2ef671a64940 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:14.692052 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c752f06-d32f-4301-a94f-32ed22db7189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:14.692682 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c752f06-d32f-4301-a94f-32ed22db7189 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:14.692964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c752f06-d32f-4301-a94f-32ed22db7189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:14.693241 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c752f06-d32f-4301-a94f-32ed22db7189 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:14.693899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c752f06-d32f-4301-a94f-32ed22db7189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:14.695594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c752f06-d32f-4301-a94f-32ed22db7189 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:14.725661 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 945/1885] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:14.727895 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017f28f-7974-49f2-9309-869988fe21f6 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:14.745133 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017f28f-7974-49f2-9309-869988fe21f6 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:14.755917 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70635b1f-6193-4273-90d3-967e9394408b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:14.756655 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70635b1f-6193-4273-90d3-967e9394408b admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:14.756944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70635b1f-6193-4273-90d3-967e9394408b admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:14.757366 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70635b1f-6193-4273-90d3-967e9394408b admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:14.758182 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70635b1f-6193-4273-90d3-967e9394408b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:14.763598 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70635b1f-6193-4273-90d3-967e9394408b admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:14.797284 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 941/1886] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:14.798447 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c752f06-d32f-4301-a94f-32ed22db7189 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:14.820338 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c752f06-d32f-4301-a94f-32ed22db7189 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:14.826934 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95696a52-6e6a-48fb-bf93-7f72e2d5664d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:14.827672 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95696a52-6e6a-48fb-bf93-7f72e2d5664d admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:14.827957 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95696a52-6e6a-48fb-bf93-7f72e2d5664d admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:14.828238 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95696a52-6e6a-48fb-bf93-7f72e2d5664d admin admin] PATH_INFO: `/v3/projects/1c97c362165a4e2dbed71989b2b37158/users/327500a37ab44a489329f849865e996f/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:14.833959 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:14.869038 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95696a52-6e6a-48fb-bf93-7f72e2d5664d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c97c362165a4e2dbed71989b2b37158, user_id=327500a37ab44a489329f849865e996f, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:14.872017 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95696a52-6e6a-48fb-bf93-7f72e2d5664d admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:14.890863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 946/1887] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:14 2022] PUT /identity/v3/projects/1c97c362165a4e2dbed71989b2b37158/users/327500a37ab44a489329f849865e996f/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:21:14.902540 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b629d04-b391-4ba0-999c-5c605a1685e6 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:14.902732 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b629d04-b391-4ba0-999c-5c605a1685e6 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:14.902966 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b629d04-b391-4ba0-999c-5c605a1685e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:14.952483 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-9b629d04-b391-4ba0-999c-5c605a1685e6 None None] MFA Rules not processed for user `327500a37ab44a489329f849865e996f`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:14.953816 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-9b629d04-b391-4ba0-999c-5c605a1685e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:15.065064 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 942/1888] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:15.069272 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95696a52-6e6a-48fb-bf93-7f72e2d5664d admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:15.155975 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95696a52-6e6a-48fb-bf93-7f72e2d5664d admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:15.162175 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e862cbb-5d01-4c01-b04e-2b5a1b6563de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:15.163233 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e862cbb-5d01-4c01-b04e-2b5a1b6563de admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:15.164681 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e862cbb-5d01-4c01-b04e-2b5a1b6563de admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:15.165255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e862cbb-5d01-4c01-b04e-2b5a1b6563de admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:15.166014 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e862cbb-5d01-4c01-b04e-2b5a1b6563de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:15.168152 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e862cbb-5d01-4c01-b04e-2b5a1b6563de admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:15.234360 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 947/1889] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:21:15 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:15.237030 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b629d04-b391-4ba0-999c-5c605a1685e6 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:15.269330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b629d04-b391-4ba0-999c-5c605a1685e6 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:15.438297 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb6ff8-7591-4fe7-b9d3-25e8fff05ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:15.440090 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb6ff8-7591-4fe7-b9d3-25e8fff05ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:15.440090 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb6ff8-7591-4fe7-b9d3-25e8fff05ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:15.440090 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb6ff8-7591-4fe7-b9d3-25e8fff05ac5 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:15.442429 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb6ff8-7591-4fe7-b9d3-25e8fff05ac5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:15.445954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb6ff8-7591-4fe7-b9d3-25e8fff05ac5 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:15.494146 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e862cbb-5d01-4c01-b04e-2b5a1b6563de admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:15.513250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 943/1890] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:21:15 2022] POST /identity/v3/users => generated 299 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:21:15.518965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e862cbb-5d01-4c01-b04e-2b5a1b6563de admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:15.543433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5fb84-f8d8-41cd-a1fd-30dee120649a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:15.544137 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5fb84-f8d8-41cd-a1fd-30dee120649a admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:15.544348 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5fb84-f8d8-41cd-a1fd-30dee120649a admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:15.544623 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5fb84-f8d8-41cd-a1fd-30dee120649a admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:15.545280 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5fb84-f8d8-41cd-a1fd-30dee120649a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:15.549476 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5fb84-f8d8-41cd-a1fd-30dee120649a admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:15.586832 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb6ff8-7591-4fe7-b9d3-25e8fff05ac5 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:15.596510 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 948/1891] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:15.600386 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb6ff8-7591-4fe7-b9d3-25e8fff05ac5 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:15.605816 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a91caa-9f77-49ed-92b0-50fbcd290383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:15.606543 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a91caa-9f77-49ed-92b0-50fbcd290383 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:15.607151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a91caa-9f77-49ed-92b0-50fbcd290383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:15.607447 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a91caa-9f77-49ed-92b0-50fbcd290383 admin admin] PATH_INFO: `/v3/projects/72bae66e0ef44e66a993a7a880489244/users/f74210bbd3de418ca42e8df90c6fd78a/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:15.612200 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:15.647330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a91caa-9f77-49ed-92b0-50fbcd290383 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72bae66e0ef44e66a993a7a880489244, user_id=f74210bbd3de418ca42e8df90c6fd78a, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:15.650788 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a91caa-9f77-49ed-92b0-50fbcd290383 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:15.675209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 944/1892] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:15 2022] PUT /identity/v3/projects/72bae66e0ef44e66a993a7a880489244/users/f74210bbd3de418ca42e8df90c6fd78a/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:15.680331 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5fb84-f8d8-41cd-a1fd-30dee120649a admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:15.693987 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5fb84-f8d8-41cd-a1fd-30dee120649a admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:15.815285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121109b2-d001-4a08-8471-fd7189ff576a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:15.815937 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121109b2-d001-4a08-8471-fd7189ff576a admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:15.816201 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121109b2-d001-4a08-8471-fd7189ff576a admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:15.816471 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121109b2-d001-4a08-8471-fd7189ff576a admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:15.817125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121109b2-d001-4a08-8471-fd7189ff576a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:15.818796 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121109b2-d001-4a08-8471-fd7189ff576a admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:15.831399 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 949/1893] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:15.835832 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a91caa-9f77-49ed-92b0-50fbcd290383 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:15.848021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a91caa-9f77-49ed-92b0-50fbcd290383 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:15.853177 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03372555-8d43-41e4-9445-57b9ff5701cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:15.853846 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03372555-8d43-41e4-9445-57b9ff5701cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:15.854265 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03372555-8d43-41e4-9445-57b9ff5701cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:15.854546 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03372555-8d43-41e4-9445-57b9ff5701cd admin admin] PATH_INFO: `/v3/projects/72bae66e0ef44e66a993a7a880489244/users/f74210bbd3de418ca42e8df90c6fd78a/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:15.856760 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03372555-8d43-41e4-9445-57b9ff5701cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=72bae66e0ef44e66a993a7a880489244, user_id=f74210bbd3de418ca42e8df90c6fd78a, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:15.862134 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03372555-8d43-41e4-9445-57b9ff5701cd admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:15.878979 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 945/1894] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:15 2022] PUT /identity/v3/projects/72bae66e0ef44e66a993a7a880489244/users/f74210bbd3de418ca42e8df90c6fd78a/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:15.890014 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4449e28-a7c4-400c-87d9-ec44e61bf0db None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:15.890302 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4449e28-a7c4-400c-87d9-ec44e61bf0db None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:15.890583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4449e28-a7c4-400c-87d9-ec44e61bf0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:15.924071 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-d4449e28-a7c4-400c-87d9-ec44e61bf0db None None] MFA Rules not processed for user `f74210bbd3de418ca42e8df90c6fd78a`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:15.925354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-d4449e28-a7c4-400c-87d9-ec44e61bf0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:16.009648 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:16.037916 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 950/1895] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:15 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:16.049029 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03372555-8d43-41e4-9445-57b9ff5701cd admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:16.057979 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-03372555-8d43-41e4-9445-57b9ff5701cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:16.135104 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03372555-8d43-41e4-9445-57b9ff5701cd admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:16.139598 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835eac57-0c98-418f-b427-6706ab2f3a0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:16.140223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835eac57-0c98-418f-b427-6706ab2f3a0f service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:16.140507 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835eac57-0c98-418f-b427-6706ab2f3a0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:16.140798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835eac57-0c98-418f-b427-6706ab2f3a0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:16.227997 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835eac57-0c98-418f-b427-6706ab2f3a0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:16.229537 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835eac57-0c98-418f-b427-6706ab2f3a0f service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:16.239206 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 946/1896] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:21:16.248922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708f6cd-ae28-410a-b815-6d4c698684d1 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:16.249383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708f6cd-ae28-410a-b815-6d4c698684d1 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:16.249540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708f6cd-ae28-410a-b815-6d4c698684d1 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:16.251409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 951/1897] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 31 00:21:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:21:16.261722 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39c29d-ff04-447a-87f7-faf5c5083f96 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:16.262021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39c29d-ff04-447a-87f7-faf5c5083f96 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:16.262338 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39c29d-ff04-447a-87f7-faf5c5083f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:16.283828 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-ac39c29d-ff04-447a-87f7-faf5c5083f96 None None] MFA Rules not processed for user `fdacdcb2451742938164fdd44cdbaebe`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:16.285672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-ac39c29d-ff04-447a-87f7-faf5c5083f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:16.427716 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 947/1898] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 31 00:21:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:16.461339 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3d225-ec98-4908-b2e6-c9015a8e77f4 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:16.461627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3d225-ec98-4908-b2e6-c9015a8e77f4 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:16.461954 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3d225-ec98-4908-b2e6-c9015a8e77f4 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:16.463684 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 952/1899] 10.222.0.67 () {54 vars in 961 bytes} [Wed Aug 31 00:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:21:16.469460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf900b-ca27-480c-9148-d610a612b441 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:16.469843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf900b-ca27-480c-9148-d610a612b441 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:16.470069 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf900b-ca27-480c-9148-d610a612b441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:16.511517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-f7bf900b-ca27-480c-9148-d610a612b441 None None] MFA Rules not processed for user `fdacdcb2451742938164fdd44cdbaebe`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:16.512907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f7bf900b-ca27-480c-9148-d610a612b441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:16.525577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 948/1900] 10.222.0.67 () {58 vars in 1058 bytes} [Wed Aug 31 00:21:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:16.538734 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3d225-ec98-4908-b2e6-c9015a8e77f4 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:16.559901 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3d225-ec98-4908-b2e6-c9015a8e77f4 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:16.575949 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5343cf1-9e28-48c7-8a8a-f0c1387b76a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdacdcb2451742938164fdd44cdbaebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:16.576789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5343cf1-9e28-48c7-8a8a-f0c1387b76a2 service placement] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:16.581292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5343cf1-9e28-48c7-8a8a-f0c1387b76a2 service placement] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:16.581292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5343cf1-9e28-48c7-8a8a-f0c1387b76a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:16.589007 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5343cf1-9e28-48c7-8a8a-f0c1387b76a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:16.590637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5343cf1-9e28-48c7-8a8a-f0c1387b76a2 service placement] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:16.615271 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 953/1901] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:21:17.030555 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf900b-ca27-480c-9148-d610a612b441 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:17.032081 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f7bf900b-ca27-480c-9148-d610a612b441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:17.054919 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:17.104061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf900b-ca27-480c-9148-d610a612b441 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:17.109249 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82664459-a853-4fa4-8291-db0d31949acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:17.109959 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82664459-a853-4fa4-8291-db0d31949acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:17.110326 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82664459-a853-4fa4-8291-db0d31949acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:17.110673 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82664459-a853-4fa4-8291-db0d31949acf admin admin] PATH_INFO: `/v3/users/c617ead6d8884fbb8499b2db118fadbc` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:17.113370 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82664459-a853-4fa4-8291-db0d31949acf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c617ead6d8884fbb8499b2db118fadbc)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:17.115531 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82664459-a853-4fa4-8291-db0d31949acf admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:17.167933 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-82664459-a853-4fa4-8291-db0d31949acf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c617ead6d8884fbb8499b2db118fadbc'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:17.180101 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-82664459-a853-4fa4-8291-db0d31949acf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c617ead6d8884fbb8499b2db118fadbc'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:17.187769 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-82664459-a853-4fa4-8291-db0d31949acf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c617ead6d8884fbb8499b2db118fadbc'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:17.197235 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-82664459-a853-4fa4-8291-db0d31949acf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c617ead6d8884fbb8499b2db118fadbc'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:17.198335 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-82664459-a853-4fa4-8291-db0d31949acf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c617ead6d8884fbb8499b2db118fadbc'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:17.205250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 949/1902] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:21:17 2022] DELETE /identity/v3/users/c617ead6d8884fbb8499b2db118fadbc => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:17.450707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5343cf1-9e28-48c7-8a8a-f0c1387b76a2 service placement] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:17.462335 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5343cf1-9e28-48c7-8a8a-f0c1387b76a2 service placement] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:17.518790 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80132a34-820f-44d0-8c63-30dde8b9764a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:17.519450 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80132a34-820f-44d0-8c63-30dde8b9764a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:17.519733 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80132a34-820f-44d0-8c63-30dde8b9764a admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:17.519999 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80132a34-820f-44d0-8c63-30dde8b9764a admin admin] PATH_INFO: `/v3/projects/b09d25d389f14922b2313ba19cb5b569` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:17.521120 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80132a34-820f-44d0-8c63-30dde8b9764a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b09d25d389f14922b2313ba19cb5b569)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:17.523613 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80132a34-820f-44d0-8c63-30dde8b9764a admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:17.568771 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-80132a34-820f-44d0-8c63-30dde8b9764a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b09d25d389f14922b2313ba19cb5b569', 'request_id': 'req-80132a34-820f-44d0-8c63-30dde8b9764a'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:17.575537 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-80132a34-820f-44d0-8c63-30dde8b9764a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b09d25d389f14922b2313ba19cb5b569', 'request_id': 'req-80132a34-820f-44d0-8c63-30dde8b9764a'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:17.581446 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-80132a34-820f-44d0-8c63-30dde8b9764a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b09d25d389f14922b2313ba19cb5b569', 'request_id': 'req-80132a34-820f-44d0-8c63-30dde8b9764a'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:17.594856 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-80132a34-820f-44d0-8c63-30dde8b9764a admin admin] The token cache is being invalidate because project b09d25d389f14922b2313ba19cb5b569 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:21:17.595182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-80132a34-820f-44d0-8c63-30dde8b9764a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:17.595801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-80132a34-820f-44d0-8c63-30dde8b9764a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:17.597280 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 954/1903] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:21:17 2022] DELETE /identity/v3/projects/b09d25d389f14922b2313ba19cb5b569 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:17.601336 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82664459-a853-4fa4-8291-db0d31949acf admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:17.603466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-82664459-a853-4fa4-8291-db0d31949acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:17.616241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82664459-a853-4fa4-8291-db0d31949acf admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:17.674457 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d69b7-a42e-4a4d-b2c7-c863036beb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:17.675111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d69b7-a42e-4a4d-b2c7-c863036beb50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:17.675394 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d69b7-a42e-4a4d-b2c7-c863036beb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:17.675666 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d69b7-a42e-4a4d-b2c7-c863036beb50 admin admin] PATH_INFO: `/v3/users/8ff416487627432ab095c864e2da8853` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:17.676780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d69b7-a42e-4a4d-b2c7-c863036beb50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ff416487627432ab095c864e2da8853)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:17.678715 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d69b7-a42e-4a4d-b2c7-c863036beb50 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:17.716506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:17.726289 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-302d69b7-a42e-4a4d-b2c7-c863036beb50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ff416487627432ab095c864e2da8853'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:17.738763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-302d69b7-a42e-4a4d-b2c7-c863036beb50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ff416487627432ab095c864e2da8853'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:17.745636 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-302d69b7-a42e-4a4d-b2c7-c863036beb50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ff416487627432ab095c864e2da8853'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:17.753002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-302d69b7-a42e-4a4d-b2c7-c863036beb50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ff416487627432ab095c864e2da8853'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:17.753660 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-302d69b7-a42e-4a4d-b2c7-c863036beb50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ff416487627432ab095c864e2da8853'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:17.760107 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 950/1904] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:21:17 2022] DELETE /identity/v3/users/8ff416487627432ab095c864e2da8853 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:17.968201 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80132a34-820f-44d0-8c63-30dde8b9764a admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:17.973148 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-80132a34-820f-44d0-8c63-30dde8b9764a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:17.992626 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80132a34-820f-44d0-8c63-30dde8b9764a admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:18.050114 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:18.057470 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5bb3d-62a7-455d-83d7-d2e7f2225e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:18.058305 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5bb3d-62a7-455d-83d7-d2e7f2225e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:18.058776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5bb3d-62a7-455d-83d7-d2e7f2225e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:18.059218 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5bb3d-62a7-455d-83d7-d2e7f2225e0b admin admin] PATH_INFO: `/v3/projects/cdc27c76fc0d4f25a85e82d8e1535828` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:18.060829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a5bb3d-62a7-455d-83d7-d2e7f2225e0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdc27c76fc0d4f25a85e82d8e1535828)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:18.063355 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a5bb3d-62a7-455d-83d7-d2e7f2225e0b admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:18.110879 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-79a5bb3d-62a7-455d-83d7-d2e7f2225e0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdc27c76fc0d4f25a85e82d8e1535828', 'request_id': 'req-79a5bb3d-62a7-455d-83d7-d2e7f2225e0b'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:18.117734 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-79a5bb3d-62a7-455d-83d7-d2e7f2225e0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdc27c76fc0d4f25a85e82d8e1535828', 'request_id': 'req-79a5bb3d-62a7-455d-83d7-d2e7f2225e0b'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:18.124135 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-79a5bb3d-62a7-455d-83d7-d2e7f2225e0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdc27c76fc0d4f25a85e82d8e1535828', 'request_id': 'req-79a5bb3d-62a7-455d-83d7-d2e7f2225e0b'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:18.141686 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-79a5bb3d-62a7-455d-83d7-d2e7f2225e0b admin admin] The token cache is being invalidate because project cdc27c76fc0d4f25a85e82d8e1535828 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:21:18.141975 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-79a5bb3d-62a7-455d-83d7-d2e7f2225e0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:18.142591 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-79a5bb3d-62a7-455d-83d7-d2e7f2225e0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:18.144099 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 955/1905] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:21:17 2022] DELETE /identity/v3/projects/cdc27c76fc0d4f25a85e82d8e1535828 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:21:18.150234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fba60a-2a4e-4b46-89a7-f80adb55bc35 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:18.150553 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fba60a-2a4e-4b46-89a7-f80adb55bc35 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:18.150830 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fba60a-2a4e-4b46-89a7-f80adb55bc35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:18.171676 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-55fba60a-2a4e-4b46-89a7-f80adb55bc35 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:18.172911 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-55fba60a-2a4e-4b46-89a7-f80adb55bc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:18.234203 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 951/1906] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:18.243128 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df5a98-76fa-43a8-9456-74fb4482f575 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:18.243300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df5a98-76fa-43a8-9456-74fb4482f575 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:18.243471 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df5a98-76fa-43a8-9456-74fb4482f575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:18.263435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-e1df5a98-76fa-43a8-9456-74fb4482f575 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:18.264710 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e1df5a98-76fa-43a8-9456-74fb4482f575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:18.273104 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 956/1907] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:18.277442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fba60a-2a4e-4b46-89a7-f80adb55bc35 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:18.287364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fba60a-2a4e-4b46-89a7-f80adb55bc35 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:18.292879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91c1a6-caa2-480a-a83b-7593406548a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:18.293480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91c1a6-caa2-480a-a83b-7593406548a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:18.293765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91c1a6-caa2-480a-a83b-7593406548a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:18.294043 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91c1a6-caa2-480a-a83b-7593406548a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:18.294690 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91c1a6-caa2-480a-a83b-7593406548a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:21:18.294997 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91c1a6-caa2-480a-a83b-7593406548a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:18.296618 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91c1a6-caa2-480a-a83b-7593406548a9 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:18.309234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 952/1908] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:21:18.313326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df5a98-76fa-43a8-9456-74fb4482f575 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:18.323509 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df5a98-76fa-43a8-9456-74fb4482f575 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:18.328090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848f6bd-f4fc-4b3a-93a8-fc9ca760c7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:18.328689 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848f6bd-f4fc-4b3a-93a8-fc9ca760c7a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:18.328971 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848f6bd-f4fc-4b3a-93a8-fc9ca760c7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:18.329245 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848f6bd-f4fc-4b3a-93a8-fc9ca760c7a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:18.329904 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848f6bd-f4fc-4b3a-93a8-fc9ca760c7a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:18.331849 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848f6bd-f4fc-4b3a-93a8-fc9ca760c7a3 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:18.359470 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 957/1909] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:21:18 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:18.363504 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91c1a6-caa2-480a-a83b-7593406548a9 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:18.374137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91c1a6-caa2-480a-a83b-7593406548a9 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:18.444782 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba81bd8-9d48-4d04-b27f-dc7edd1f4669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:18.445393 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba81bd8-9d48-4d04-b27f-dc7edd1f4669 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:18.445671 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba81bd8-9d48-4d04-b27f-dc7edd1f4669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:18.445951 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba81bd8-9d48-4d04-b27f-dc7edd1f4669 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:18.446735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba81bd8-9d48-4d04-b27f-dc7edd1f4669 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:18.448684 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba81bd8-9d48-4d04-b27f-dc7edd1f4669 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:18.470903 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 953/1910] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:21:18 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:18.475068 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848f6bd-f4fc-4b3a-93a8-fc9ca760c7a3 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:18.486935 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848f6bd-f4fc-4b3a-93a8-fc9ca760c7a3 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:18.492210 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8df683-258b-4c49-96bc-3335490cc71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:18.492978 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8df683-258b-4c49-96bc-3335490cc71c admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:18.493263 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8df683-258b-4c49-96bc-3335490cc71c admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:18.493542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8df683-258b-4c49-96bc-3335490cc71c admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:18.494193 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8df683-258b-4c49-96bc-3335490cc71c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:18.495867 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8df683-258b-4c49-96bc-3335490cc71c admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:18.506288 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 958/1911] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:18.510360 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba81bd8-9d48-4d04-b27f-dc7edd1f4669 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:18.521003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba81bd8-9d48-4d04-b27f-dc7edd1f4669 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:18.526257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7da737-0746-417a-91c2-544e45307775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:18.526871 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7da737-0746-417a-91c2-544e45307775 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:18.527330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7da737-0746-417a-91c2-544e45307775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:18.527612 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7da737-0746-417a-91c2-544e45307775 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:18.528258 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7da737-0746-417a-91c2-544e45307775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:18.529852 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7da737-0746-417a-91c2-544e45307775 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:18.542493 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 954/1912] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:18.546597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8df683-258b-4c49-96bc-3335490cc71c admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:18.557968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8df683-258b-4c49-96bc-3335490cc71c admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:18.562741 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b10c2-0396-4381-9344-111bf42000e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:18.563443 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b10c2-0396-4381-9344-111bf42000e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:18.563725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b10c2-0396-4381-9344-111bf42000e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:18.563993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b10c2-0396-4381-9344-111bf42000e2 admin admin] PATH_INFO: `/v3/projects/ade91b4ec7ae48b78f4616fbdbf253a9/users/3c58b3edba70494aac0695c66f4eacd9/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:18.580783 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b10c2-0396-4381-9344-111bf42000e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ade91b4ec7ae48b78f4616fbdbf253a9, user_id=3c58b3edba70494aac0695c66f4eacd9, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:18.583489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b10c2-0396-4381-9344-111bf42000e2 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:18.597153 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 959/1913] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:18 2022] PUT /identity/v3/projects/ade91b4ec7ae48b78f4616fbdbf253a9/users/3c58b3edba70494aac0695c66f4eacd9/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:21:18.606728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50feb5d1-a721-4e34-a417-375371d94ad2 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:18.607677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50feb5d1-a721-4e34-a417-375371d94ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:18.607962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50feb5d1-a721-4e34-a417-375371d94ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:18.635792 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-50feb5d1-a721-4e34-a417-375371d94ad2 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:18.637377 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-50feb5d1-a721-4e34-a417-375371d94ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:18.652267 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:18.703393 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 955/1914] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:18.713571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b10c2-0396-4381-9344-111bf42000e2 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:18.715301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-289b10c2-0396-4381-9344-111bf42000e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:18.771165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b10c2-0396-4381-9344-111bf42000e2 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:18.775440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcc1549-337a-4d67-a490-6e76bc0decb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:18.776082 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc1549-337a-4d67-a490-6e76bc0decb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:18.776359 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc1549-337a-4d67-a490-6e76bc0decb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:18.776630 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc1549-337a-4d67-a490-6e76bc0decb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:18.787651 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcc1549-337a-4d67-a490-6e76bc0decb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:18.789892 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcc1549-337a-4d67-a490-6e76bc0decb0 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:18.799456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 960/1915] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:21:19.517800 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a8236-6df3-4375-a3e0-dabbfda50880 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:19.518336 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a8236-6df3-4375-a3e0-dabbfda50880 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:19.518958 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a8236-6df3-4375-a3e0-dabbfda50880 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:19.521122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 956/1916] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 31 00:21:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:21:19.535447 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fb363-428e-4961-873a-a101439e3922 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:19.536044 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fb363-428e-4961-873a-a101439e3922 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:19.536044 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fb363-428e-4961-873a-a101439e3922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:19.571604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-811fb363-428e-4961-873a-a101439e3922 None None] MFA Rules not processed for user `3aa53ae5a30146f69d7d6eefe4290dba`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:19.574356 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-811fb363-428e-4961-873a-a101439e3922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:19.606328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:19.711369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 961/1917] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 31 00:21:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:20.153672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a8236-6df3-4375-a3e0-dabbfda50880 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:20.157626 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-2f0a8236-6df3-4375-a3e0-dabbfda50880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:20.191918 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a8236-6df3-4375-a3e0-dabbfda50880 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:20.199305 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d6488-34ab-4241-9074-5ef316bedbc6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:20.200295 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d6488-34ab-4241-9074-5ef316bedbc6 service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:20.200939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d6488-34ab-4241-9074-5ef316bedbc6 service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:20.201469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d6488-34ab-4241-9074-5ef316bedbc6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:20.214368 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d6488-34ab-4241-9074-5ef316bedbc6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:20.217374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d6488-34ab-4241-9074-5ef316bedbc6 service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:20.245116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 957/1918] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:21:20.490005 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fb363-428e-4961-873a-a101439e3922 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:20.492368 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-811fb363-428e-4961-873a-a101439e3922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:20.506885 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fb363-428e-4961-873a-a101439e3922 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:20.512014 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bb692-76e2-411f-a48d-da9af4d46d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:20.512670 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bb692-76e2-411f-a48d-da9af4d46d7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:20.512951 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bb692-76e2-411f-a48d-da9af4d46d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:20.513228 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bb692-76e2-411f-a48d-da9af4d46d7d admin admin] PATH_INFO: `/v3/users/4f362d1d375747cda66f8b7bbdc2bd95` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:20.514432 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6bb692-76e2-411f-a48d-da9af4d46d7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f362d1d375747cda66f8b7bbdc2bd95)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:20.516647 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6bb692-76e2-411f-a48d-da9af4d46d7d admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:20.568264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-0d6bb692-76e2-411f-a48d-da9af4d46d7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f362d1d375747cda66f8b7bbdc2bd95'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:20.576800 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-0d6bb692-76e2-411f-a48d-da9af4d46d7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f362d1d375747cda66f8b7bbdc2bd95'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:20.584307 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-0d6bb692-76e2-411f-a48d-da9af4d46d7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f362d1d375747cda66f8b7bbdc2bd95'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:20.585046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-0d6bb692-76e2-411f-a48d-da9af4d46d7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f362d1d375747cda66f8b7bbdc2bd95'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:20.586413 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-0d6bb692-76e2-411f-a48d-da9af4d46d7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f362d1d375747cda66f8b7bbdc2bd95'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:20.606200 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 962/1919] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:21:20 2022] DELETE /identity/v3/users/4f362d1d375747cda66f8b7bbdc2bd95 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:20.610678 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff5cff-99cb-43bf-81c9-99d8c5379b1d None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:20.610964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff5cff-99cb-43bf-81c9-99d8c5379b1d None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:20.611307 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff5cff-99cb-43bf-81c9-99d8c5379b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:20.630807 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-f0ff5cff-99cb-43bf-81c9-99d8c5379b1d None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:20.632052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f0ff5cff-99cb-43bf-81c9-99d8c5379b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:20.707876 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 958/1920] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:20.718305 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bb692-76e2-411f-a48d-da9af4d46d7d admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:20.719926 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-0d6bb692-76e2-411f-a48d-da9af4d46d7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:20.805142 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bb692-76e2-411f-a48d-da9af4d46d7d admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:20.809256 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38806cf1-94bc-47ac-8e50-d742cb352af6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:20.809901 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38806cf1-94bc-47ac-8e50-d742cb352af6 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:20.810178 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38806cf1-94bc-47ac-8e50-d742cb352af6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:20.810599 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38806cf1-94bc-47ac-8e50-d742cb352af6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:20.822943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f05daf-f604-4ce0-8e8d-94bcbdad79d9 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:20.823271 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f05daf-f604-4ce0-8e8d-94bcbdad79d9 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:20.823730 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f05daf-f604-4ce0-8e8d-94bcbdad79d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:20.833788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38806cf1-94bc-47ac-8e50-d742cb352af6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:20.835998 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38806cf1-94bc-47ac-8e50-d742cb352af6 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:20.846535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 963/1921] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:21:20.876372 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-12f05daf-f604-4ce0-8e8d-94bcbdad79d9 None None] MFA Rules not processed for user `3c58b3edba70494aac0695c66f4eacd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:20.882062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-12f05daf-f604-4ce0-8e8d-94bcbdad79d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:20.999081 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 959/1922] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:21:21.001522 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38806cf1-94bc-47ac-8e50-d742cb352af6 service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:21.006422 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-38806cf1-94bc-47ac-8e50-d742cb352af6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:21.019461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38806cf1-94bc-47ac-8e50-d742cb352af6 service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:21.024734 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a4971-dd88-448b-814e-5953fc131513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:21.025344 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a4971-dd88-448b-814e-5953fc131513 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:21.025629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a4971-dd88-448b-814e-5953fc131513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:21.025915 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a4971-dd88-448b-814e-5953fc131513 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:21.026529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a4971-dd88-448b-814e-5953fc131513 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:21.028465 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a4971-dd88-448b-814e-5953fc131513 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:21.070167 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 964/1923] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:21:21 2022] POST /identity/v3/projects => generated 371 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:21.074382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f05daf-f604-4ce0-8e8d-94bcbdad79d9 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:21.086569 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f05daf-f604-4ce0-8e8d-94bcbdad79d9 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:21.122729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:21.147024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c364762-1006-4acf-a450-b5ac09ccf4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:21.147781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c364762-1006-4acf-a450-b5ac09ccf4b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:21.148730 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c364762-1006-4acf-a450-b5ac09ccf4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:21.149019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c364762-1006-4acf-a450-b5ac09ccf4b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:21.149879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c364762-1006-4acf-a450-b5ac09ccf4b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:21.152230 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c364762-1006-4acf-a450-b5ac09ccf4b9 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:21.183246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 960/1924] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:21:21 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:21.184048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a4971-dd88-448b-814e-5953fc131513 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:21.184201 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c364762-1006-4acf-a450-b5ac09ccf4b9 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:21.185864 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-d94a4971-dd88-448b-814e-5953fc131513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:21.198406 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c364762-1006-4acf-a450-b5ac09ccf4b9 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:21.203997 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2da4a-b765-4497-a638-b99f706f814d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:21.204908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2da4a-b765-4497-a638-b99f706f814d admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:21.205072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2da4a-b765-4497-a638-b99f706f814d admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:21.205333 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2da4a-b765-4497-a638-b99f706f814d admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:21.206095 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb2da4a-b765-4497-a638-b99f706f814d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:21.207626 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb2da4a-b765-4497-a638-b99f706f814d admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:21.223965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a4971-dd88-448b-814e-5953fc131513 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:21.224615 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 961/1925] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:21.232369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2da4a-b765-4497-a638-b99f706f814d admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:21.235460 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad7e66-eb1d-4afa-af39-264f59ee2277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:21.235460 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad7e66-eb1d-4afa-af39-264f59ee2277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:21.235460 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad7e66-eb1d-4afa-af39-264f59ee2277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:21.235460 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad7e66-eb1d-4afa-af39-264f59ee2277 admin admin] PATH_INFO: `/v3/projects/d51b981cd6a6411c9d4c0fecd834db22` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:21.237599 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad7e66-eb1d-4afa-af39-264f59ee2277 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d51b981cd6a6411c9d4c0fecd834db22)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:21.244373 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2da4a-b765-4497-a638-b99f706f814d admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:21.245873 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad7e66-eb1d-4afa-af39-264f59ee2277 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:21.249329 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb387e2b-e83f-4294-b07d-679cf97a8c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:21.250004 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb387e2b-e83f-4294-b07d-679cf97a8c9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:21.250285 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb387e2b-e83f-4294-b07d-679cf97a8c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:21.250560 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb387e2b-e83f-4294-b07d-679cf97a8c9c admin admin] PATH_INFO: `/v3/projects/a990dc1179c141df8082f961a8de8d42/users/ecd42f6e02954a3f917cd8a1fe00f7aa/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:21.254806 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:21.269641 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb387e2b-e83f-4294-b07d-679cf97a8c9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a990dc1179c141df8082f961a8de8d42, user_id=ecd42f6e02954a3f917cd8a1fe00f7aa, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:21.272694 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb387e2b-e83f-4294-b07d-679cf97a8c9c admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:21.291499 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 962/1926] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:21 2022] PUT /identity/v3/projects/a990dc1179c141df8082f961a8de8d42/users/ecd42f6e02954a3f917cd8a1fe00f7aa/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:21.295635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb387e2b-e83f-4294-b07d-679cf97a8c9c admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:21.331595 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d1ad7e66-eb1d-4afa-af39-264f59ee2277 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd51b981cd6a6411c9d4c0fecd834db22', 'request_id': 'req-d1ad7e66-eb1d-4afa-af39-264f59ee2277'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:21.346517 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d1ad7e66-eb1d-4afa-af39-264f59ee2277 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd51b981cd6a6411c9d4c0fecd834db22', 'request_id': 'req-d1ad7e66-eb1d-4afa-af39-264f59ee2277'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:21.355840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d1ad7e66-eb1d-4afa-af39-264f59ee2277 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd51b981cd6a6411c9d4c0fecd834db22', 'request_id': 'req-d1ad7e66-eb1d-4afa-af39-264f59ee2277'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:21.372227 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d1ad7e66-eb1d-4afa-af39-264f59ee2277 admin admin] The token cache is being invalidate because project d51b981cd6a6411c9d4c0fecd834db22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:21:21.372686 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d1ad7e66-eb1d-4afa-af39-264f59ee2277 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:21.373495 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d1ad7e66-eb1d-4afa-af39-264f59ee2277 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:21.375947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 965/1927] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:21:21 2022] DELETE /identity/v3/projects/d51b981cd6a6411c9d4c0fecd834db22 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:21:21.378425 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb387e2b-e83f-4294-b07d-679cf97a8c9c admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:21.379653 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad7e66-eb1d-4afa-af39-264f59ee2277 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:21.381539 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-d1ad7e66-eb1d-4afa-af39-264f59ee2277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:21.471806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19caca14-0eff-4826-91c6-843000607661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:21.472491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19caca14-0eff-4826-91c6-843000607661 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:21.472770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19caca14-0eff-4826-91c6-843000607661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:21.473045 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19caca14-0eff-4826-91c6-843000607661 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:21.473688 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19caca14-0eff-4826-91c6-843000607661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:21.475311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19caca14-0eff-4826-91c6-843000607661 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:21.477366 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad7e66-eb1d-4afa-af39-264f59ee2277 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:21.482767 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e92ec-3522-46eb-b61b-4a1e7fa9f1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:21.483467 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e92ec-3522-46eb-b61b-4a1e7fa9f1b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:21.483776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e92ec-3522-46eb-b61b-4a1e7fa9f1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:21.484052 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e92ec-3522-46eb-b61b-4a1e7fa9f1b1 admin admin] PATH_INFO: `/v3/users/31dfbeae8ea84958aee5a69280449c6f` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:21.485258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e92ec-3522-46eb-b61b-4a1e7fa9f1b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31dfbeae8ea84958aee5a69280449c6f)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:21.486096 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 963/1928] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:21:21.487614 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e92ec-3522-46eb-b61b-4a1e7fa9f1b1 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:21.490208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19caca14-0eff-4826-91c6-843000607661 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:21.492352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-19caca14-0eff-4826-91c6-843000607661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:21.505105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19caca14-0eff-4826-91c6-843000607661 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:21.510427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d9ba9-1bbc-4918-ba94-b5b09cdab986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:21.511400 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d9ba9-1bbc-4918-ba94-b5b09cdab986 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:21.511675 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d9ba9-1bbc-4918-ba94-b5b09cdab986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:21.512081 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d9ba9-1bbc-4918-ba94-b5b09cdab986 admin admin] PATH_INFO: `/v3/projects/a990dc1179c141df8082f961a8de8d42/users/ecd42f6e02954a3f917cd8a1fe00f7aa/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:21.514124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d9ba9-1bbc-4918-ba94-b5b09cdab986 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a990dc1179c141df8082f961a8de8d42, user_id=ecd42f6e02954a3f917cd8a1fe00f7aa, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:21.516835 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d9ba9-1bbc-4918-ba94-b5b09cdab986 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:21.528084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 964/1929] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:21 2022] PUT /identity/v3/projects/a990dc1179c141df8082f961a8de8d42/users/ecd42f6e02954a3f917cd8a1fe00f7aa/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:21:21.538221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-718e92ec-3522-46eb-b61b-4a1e7fa9f1b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31dfbeae8ea84958aee5a69280449c6f'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:21.546156 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-718e92ec-3522-46eb-b61b-4a1e7fa9f1b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31dfbeae8ea84958aee5a69280449c6f'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:21.553518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-718e92ec-3522-46eb-b61b-4a1e7fa9f1b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31dfbeae8ea84958aee5a69280449c6f'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:21.554412 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-718e92ec-3522-46eb-b61b-4a1e7fa9f1b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31dfbeae8ea84958aee5a69280449c6f'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:21.555138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-718e92ec-3522-46eb-b61b-4a1e7fa9f1b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31dfbeae8ea84958aee5a69280449c6f'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:21.574682 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 966/1930] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:21:21 2022] DELETE /identity/v3/users/31dfbeae8ea84958aee5a69280449c6f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:21.875777 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d9ba9-1bbc-4918-ba94-b5b09cdab986 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:21.877621 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-6a7d9ba9-1bbc-4918-ba94-b5b09cdab986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:21.928542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:22.005417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d9ba9-1bbc-4918-ba94-b5b09cdab986 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:22.091919 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f030a0-2d28-43b4-9d8f-6e62d410ae57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:22.092649 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f030a0-2d28-43b4-9d8f-6e62d410ae57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:22.092937 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f030a0-2d28-43b4-9d8f-6e62d410ae57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:22.093212 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f030a0-2d28-43b4-9d8f-6e62d410ae57 admin admin] PATH_INFO: `/v3/projects/2b571aadc71c4a36822a99f995203e63` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:22.094361 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f030a0-2d28-43b4-9d8f-6e62d410ae57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b571aadc71c4a36822a99f995203e63)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:22.096618 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f030a0-2d28-43b4-9d8f-6e62d410ae57 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:22.160056 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-d1f030a0-2d28-43b4-9d8f-6e62d410ae57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b571aadc71c4a36822a99f995203e63', 'request_id': 'req-d1f030a0-2d28-43b4-9d8f-6e62d410ae57'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:22.170426 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-d1f030a0-2d28-43b4-9d8f-6e62d410ae57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b571aadc71c4a36822a99f995203e63', 'request_id': 'req-d1f030a0-2d28-43b4-9d8f-6e62d410ae57'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:22.177726 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-d1f030a0-2d28-43b4-9d8f-6e62d410ae57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b571aadc71c4a36822a99f995203e63', 'request_id': 'req-d1f030a0-2d28-43b4-9d8f-6e62d410ae57'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:22.191957 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-d1f030a0-2d28-43b4-9d8f-6e62d410ae57 admin admin] The token cache is being invalidate because project 2b571aadc71c4a36822a99f995203e63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:21:22.192597 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-d1f030a0-2d28-43b4-9d8f-6e62d410ae57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:22.192906 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-d1f030a0-2d28-43b4-9d8f-6e62d410ae57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:22.200640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 965/1931] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:21:21 2022] DELETE /identity/v3/projects/2b571aadc71c4a36822a99f995203e63 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:22.202468 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cead5d-a017-4a56-87af-44705a408f26 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:22.202767 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cead5d-a017-4a56-87af-44705a408f26 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:22.203767 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cead5d-a017-4a56-87af-44705a408f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:22.228297 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-c5cead5d-a017-4a56-87af-44705a408f26 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:22.229526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-c5cead5d-a017-4a56-87af-44705a408f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:22.240761 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:22.268591 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0b3d0-4906-4018-8098-8db66da5262d None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:22.269360 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0b3d0-4906-4018-8098-8db66da5262d None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:22.269360 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0b3d0-4906-4018-8098-8db66da5262d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:22.300125 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-65d0b3d0-4906-4018-8098-8db66da5262d None None] MFA Rules not processed for user `ecd42f6e02954a3f917cd8a1fe00f7aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:22.301704 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-65d0b3d0-4906-4018-8098-8db66da5262d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:22.310679 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 967/1932] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:22.318628 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073c3d4-e37d-4111-a68c-e8de79d1fca5 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:22.319026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073c3d4-e37d-4111-a68c-e8de79d1fca5 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:22.319334 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073c3d4-e37d-4111-a68c-e8de79d1fca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:22.342738 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-e073c3d4-e37d-4111-a68c-e8de79d1fca5 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:22.344007 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e073c3d4-e37d-4111-a68c-e8de79d1fca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:22.352255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 968/1933] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:21:22.356498 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e073c3d4-e37d-4111-a68c-e8de79d1fca5 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:22.367585 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:22.371085 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e073c3d4-e37d-4111-a68c-e8de79d1fca5 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:22.376866 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f28e54-19dc-4d94-9129-f531c1d14a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:22.377826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f28e54-19dc-4d94-9129-f531c1d14a34 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:22.378008 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f28e54-19dc-4d94-9129-f531c1d14a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:22.378234 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f28e54-19dc-4d94-9129-f531c1d14a34 admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:22.379322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f28e54-19dc-4d94-9129-f531c1d14a34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:21:22.379428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f28e54-19dc-4d94-9129-f531c1d14a34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:22.381284 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f28e54-19dc-4d94-9129-f531c1d14a34 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:22.400741 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 969/1935] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:21:22.400928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 966/1935] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:22 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:21:22.402091 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f28e54-19dc-4d94-9129-f531c1d14a34 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:22.408708 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d0b3d0-4906-4018-8098-8db66da5262d None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:22.410107 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-65d0b3d0-4906-4018-8098-8db66da5262d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:22.413244 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f28e54-19dc-4d94-9129-f531c1d14a34 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:22.419654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3190ba-47ee-4209-999f-1a9e84fd6266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:22.420397 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3190ba-47ee-4209-999f-1a9e84fd6266 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:22.420750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3190ba-47ee-4209-999f-1a9e84fd6266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:22.421092 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3190ba-47ee-4209-999f-1a9e84fd6266 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:22.421807 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3190ba-47ee-4209-999f-1a9e84fd6266 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:22.423886 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3190ba-47ee-4209-999f-1a9e84fd6266 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:22.461576 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 970/1936] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:21:22 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:22.462883 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3190ba-47ee-4209-999f-1a9e84fd6266 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:22.512030 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d0b3d0-4906-4018-8098-8db66da5262d None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:22.543146 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3190ba-47ee-4209-999f-1a9e84fd6266 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:22.543146 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6e1da-42b6-487f-9a94-f6bd33d50127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:22.543146 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6e1da-42b6-487f-9a94-f6bd33d50127 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:22.543146 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6e1da-42b6-487f-9a94-f6bd33d50127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:22.543656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6e1da-42b6-487f-9a94-f6bd33d50127 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:22.543656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f6e1da-42b6-487f-9a94-f6bd33d50127 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:22.544180 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f6e1da-42b6-487f-9a94-f6bd33d50127 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:22.576166 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 971/1937] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:21:22 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:22.582325 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6e1da-42b6-487f-9a94-f6bd33d50127 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:22.597373 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6e1da-42b6-487f-9a94-f6bd33d50127 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:22.597983 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d77db-11f7-45af-985c-0b537e6802a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:22.598803 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d77db-11f7-45af-985c-0b537e6802a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:22.602149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d77db-11f7-45af-985c-0b537e6802a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:22.602289 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d77db-11f7-45af-985c-0b537e6802a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:22.603585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a4848-1044-4f11-ba39-ecc2ea1e1281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:22.604223 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a4848-1044-4f11-ba39-ecc2ea1e1281 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:22.604505 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a4848-1044-4f11-ba39-ecc2ea1e1281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:22.604783 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a4848-1044-4f11-ba39-ecc2ea1e1281 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:22.605447 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a4848-1044-4f11-ba39-ecc2ea1e1281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:22.606173 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-d07d77db-11f7-45af-985c-0b537e6802a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:22.607099 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a4848-1044-4f11-ba39-ecc2ea1e1281 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:22.621826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 972/1938] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:21:22.623244 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a4848-1044-4f11-ba39-ecc2ea1e1281 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:22.640764 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a4848-1044-4f11-ba39-ecc2ea1e1281 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:22.648999 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958699b-2764-4ced-9813-80005a277d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:22.649876 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958699b-2764-4ced-9813-80005a277d5a admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:22.650164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958699b-2764-4ced-9813-80005a277d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:22.650590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958699b-2764-4ced-9813-80005a277d5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:22.651252 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d958699b-2764-4ced-9813-80005a277d5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:22.653587 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d958699b-2764-4ced-9813-80005a277d5a admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:22.668553 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 973/1939] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:22.673547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958699b-2764-4ced-9813-80005a277d5a admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:22.687637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958699b-2764-4ced-9813-80005a277d5a admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:22.693589 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b93f4-7c6a-4e96-9600-bc62a8c720e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:22.694229 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b93f4-7c6a-4e96-9600-bc62a8c720e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:22.695054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b93f4-7c6a-4e96-9600-bc62a8c720e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:22.695413 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b93f4-7c6a-4e96-9600-bc62a8c720e1 admin admin] PATH_INFO: `/v3/projects/f90b497e881e4b9e8433b308057645bf/users/eaada3ab06fd4e38b1a9d71a1981f7d1/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:22.704289 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d77db-11f7-45af-985c-0b537e6802a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:22.706282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d77db-11f7-45af-985c-0b537e6802a3 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:22.714114 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b93f4-7c6a-4e96-9600-bc62a8c720e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f90b497e881e4b9e8433b308057645bf, user_id=eaada3ab06fd4e38b1a9d71a1981f7d1, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:22.717776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b93f4-7c6a-4e96-9600-bc62a8c720e1 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:22.736743 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 967/1940] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:21:22.736864 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 974/1941] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:22 2022] PUT /identity/v3/projects/f90b497e881e4b9e8433b308057645bf/users/eaada3ab06fd4e38b1a9d71a1981f7d1/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:22.749821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b5894-7e3b-4756-8783-d10c3872ce95 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:22.750113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b5894-7e3b-4756-8783-d10c3872ce95 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:22.750394 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b5894-7e3b-4756-8783-d10c3872ce95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:22.782181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-7f4b5894-7e3b-4756-8783-d10c3872ce95 None None] MFA Rules not processed for user `eaada3ab06fd4e38b1a9d71a1981f7d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:22.783431 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-7f4b5894-7e3b-4756-8783-d10c3872ce95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:22.826539 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:22.850569 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 968/1942] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:22.855617 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b93f4-7c6a-4e96-9600-bc62a8c720e1 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:22.873675 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:22.924375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b93f4-7c6a-4e96-9600-bc62a8c720e1 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:22.929910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db0699-4972-4a34-8255-f6c737b0db8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:22.930928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db0699-4972-4a34-8255-f6c737b0db8b admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:22.931393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db0699-4972-4a34-8255-f6c737b0db8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:22.931798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db0699-4972-4a34-8255-f6c737b0db8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:22.932807 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db0699-4972-4a34-8255-f6c737b0db8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:22.936239 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db0699-4972-4a34-8255-f6c737b0db8b admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:22.975984 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 975/1943] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:21:22 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:22.980508 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b5894-7e3b-4756-8783-d10c3872ce95 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:22.993110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b5894-7e3b-4756-8783-d10c3872ce95 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:23.058404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23931df-4439-4fd2-ab75-6581bc4e3c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:23.059250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23931df-4439-4fd2-ab75-6581bc4e3c7a admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:23.059610 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23931df-4439-4fd2-ab75-6581bc4e3c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:23.059884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23931df-4439-4fd2-ab75-6581bc4e3c7a admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:23.060519 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23931df-4439-4fd2-ab75-6581bc4e3c7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:23.062688 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23931df-4439-4fd2-ab75-6581bc4e3c7a admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:23.087050 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 969/1944] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:21:22 2022] POST /identity/v3/users => generated 299 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:23.091135 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db0699-4972-4a34-8255-f6c737b0db8b admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:23.101798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db0699-4972-4a34-8255-f6c737b0db8b admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:23.110495 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c4809-c905-4111-9dd2-2a2733fe0ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:23.111179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c4809-c905-4111-9dd2-2a2733fe0ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:23.111441 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c4809-c905-4111-9dd2-2a2733fe0ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:23.111709 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c4809-c905-4111-9dd2-2a2733fe0ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:23.112363 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c4809-c905-4111-9dd2-2a2733fe0ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:23.114081 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c4809-c905-4111-9dd2-2a2733fe0ce8 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:23.131224 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 976/1945] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:23.131870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23931df-4439-4fd2-ab75-6581bc4e3c7a admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:23.148249 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23931df-4439-4fd2-ab75-6581bc4e3c7a admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:23.153599 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af4c72-a691-4a89-a164-f8355c27498b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:23.154206 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af4c72-a691-4a89-a164-f8355c27498b admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:23.154491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af4c72-a691-4a89-a164-f8355c27498b admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:23.155240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af4c72-a691-4a89-a164-f8355c27498b admin admin] PATH_INFO: `/v3/projects/90daa297b5384564bcdae8c92acc9672/users/ef1daf57da074905bed6497ce0c67887/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:23.173769 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0af4c72-a691-4a89-a164-f8355c27498b admin admin] RBAC: Authorizing `identity:create_grant(project_id=90daa297b5384564bcdae8c92acc9672, user_id=ef1daf57da074905bed6497ce0c67887, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:23.176609 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0af4c72-a691-4a89-a164-f8355c27498b admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:23.192679 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 970/1946] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:23 2022] PUT /identity/v3/projects/90daa297b5384564bcdae8c92acc9672/users/ef1daf57da074905bed6497ce0c67887/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:21:23.197179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c4809-c905-4111-9dd2-2a2733fe0ce8 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:23.208848 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c4809-c905-4111-9dd2-2a2733fe0ce8 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:23.276611 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36740b66-bc81-4c15-a103-99fe9bc04817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:23.277238 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36740b66-bc81-4c15-a103-99fe9bc04817 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:23.277519 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36740b66-bc81-4c15-a103-99fe9bc04817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:23.277794 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36740b66-bc81-4c15-a103-99fe9bc04817 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:23.278450 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36740b66-bc81-4c15-a103-99fe9bc04817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:23.280130 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36740b66-bc81-4c15-a103-99fe9bc04817 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:23.291688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 977/1947] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:23.296029 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af4c72-a691-4a89-a164-f8355c27498b admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:23.307026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af4c72-a691-4a89-a164-f8355c27498b admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:23.312076 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c924c7ce-42ce-48d7-ab92-04d1116fc26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:23.312900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924c7ce-42ce-48d7-ab92-04d1116fc26a admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:23.313174 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924c7ce-42ce-48d7-ab92-04d1116fc26a admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:23.313444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924c7ce-42ce-48d7-ab92-04d1116fc26a admin admin] PATH_INFO: `/v3/projects/90daa297b5384564bcdae8c92acc9672/users/ef1daf57da074905bed6497ce0c67887/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:23.315380 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c924c7ce-42ce-48d7-ab92-04d1116fc26a admin admin] RBAC: Authorizing `identity:create_grant(project_id=90daa297b5384564bcdae8c92acc9672, user_id=ef1daf57da074905bed6497ce0c67887, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:23.318001 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c924c7ce-42ce-48d7-ab92-04d1116fc26a admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:23.330337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 971/1948] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:23 2022] PUT /identity/v3/projects/90daa297b5384564bcdae8c92acc9672/users/ef1daf57da074905bed6497ce0c67887/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:21:23.339729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52565eb-c102-4cf2-a0e5-8da3f7a70ae3 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:23.340014 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52565eb-c102-4cf2-a0e5-8da3f7a70ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:23.340294 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52565eb-c102-4cf2-a0e5-8da3f7a70ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:23.375513 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-d52565eb-c102-4cf2-a0e5-8da3f7a70ae3 None None] MFA Rules not processed for user `ef1daf57da074905bed6497ce0c67887`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:23.376728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-d52565eb-c102-4cf2-a0e5-8da3f7a70ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:23.446807 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 978/1949] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:23 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:23.463918 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c924c7ce-42ce-48d7-ab92-04d1116fc26a admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:23.465647 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c924c7ce-42ce-48d7-ab92-04d1116fc26a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:23.524634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c924c7ce-42ce-48d7-ab92-04d1116fc26a admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:23.528385 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457a164-b8fd-4f46-8261-2dc9efb17694 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:23.528991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457a164-b8fd-4f46-8261-2dc9efb17694 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:23.529272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457a164-b8fd-4f46-8261-2dc9efb17694 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:23.529546 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457a164-b8fd-4f46-8261-2dc9efb17694 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:23.538664 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457a164-b8fd-4f46-8261-2dc9efb17694 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:23.540365 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457a164-b8fd-4f46-8261-2dc9efb17694 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:23.549830 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 972/1950] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:23 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:21:24.166801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52565eb-c102-4cf2-a0e5-8da3f7a70ae3 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:24.180031 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:24.233853 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52565eb-c102-4cf2-a0e5-8da3f7a70ae3 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:24.238563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3de7bc-e732-4cd3-8857-af4951067d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:24.239228 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3de7bc-e732-4cd3-8857-af4951067d18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:24.239511 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3de7bc-e732-4cd3-8857-af4951067d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:24.239784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3de7bc-e732-4cd3-8857-af4951067d18 admin admin] PATH_INFO: `/v3/users/eaada3ab06fd4e38b1a9d71a1981f7d1` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:24.241256 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3de7bc-e732-4cd3-8857-af4951067d18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaada3ab06fd4e38b1a9d71a1981f7d1)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:24.243909 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3de7bc-e732-4cd3-8857-af4951067d18 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:24.290175 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3d3de7bc-e732-4cd3-8857-af4951067d18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaada3ab06fd4e38b1a9d71a1981f7d1'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:24.297583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3d3de7bc-e732-4cd3-8857-af4951067d18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaada3ab06fd4e38b1a9d71a1981f7d1'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:24.305540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3d3de7bc-e732-4cd3-8857-af4951067d18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaada3ab06fd4e38b1a9d71a1981f7d1'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:24.306613 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3d3de7bc-e732-4cd3-8857-af4951067d18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaada3ab06fd4e38b1a9d71a1981f7d1'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:24.307271 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3d3de7bc-e732-4cd3-8857-af4951067d18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaada3ab06fd4e38b1a9d71a1981f7d1'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:24.325916 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 979/1951] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:21:24 2022] DELETE /identity/v3/users/eaada3ab06fd4e38b1a9d71a1981f7d1 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:24.330620 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62740e05-4c5d-412d-b730-7d2d8c3c02fd None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:24.331151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62740e05-4c5d-412d-b730-7d2d8c3c02fd None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:24.331243 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62740e05-4c5d-412d-b730-7d2d8c3c02fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:24.349700 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-62740e05-4c5d-412d-b730-7d2d8c3c02fd None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:24.350878 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-62740e05-4c5d-412d-b730-7d2d8c3c02fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:24.384937 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:24.411911 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 973/1952] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:24.422413 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3de7bc-e732-4cd3-8857-af4951067d18 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:24.423876 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-3d3de7bc-e732-4cd3-8857-af4951067d18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:24.484977 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3de7bc-e732-4cd3-8857-af4951067d18 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:24.488802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5393c-f948-42d8-877a-7289b5b92d8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:24.489413 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5393c-f948-42d8-877a-7289b5b92d8f service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:24.489692 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5393c-f948-42d8-877a-7289b5b92d8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:24.489962 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5393c-f948-42d8-877a-7289b5b92d8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:24.502101 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d5393c-f948-42d8-877a-7289b5b92d8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:24.503582 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d5393c-f948-42d8-877a-7289b5b92d8f service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:24.514091 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 980/1953] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:21:24.947101 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62740e05-4c5d-412d-b730-7d2d8c3c02fd None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:24.948564 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-62740e05-4c5d-412d-b730-7d2d8c3c02fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:24.963669 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62740e05-4c5d-412d-b730-7d2d8c3c02fd None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:24.974219 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77536cf3-2ce2-4112-b2d8-bfd3311be1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:24.974219 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77536cf3-2ce2-4112-b2d8-bfd3311be1cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:24.974219 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77536cf3-2ce2-4112-b2d8-bfd3311be1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:24.974219 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77536cf3-2ce2-4112-b2d8-bfd3311be1cc admin admin] PATH_INFO: `/v3/projects/f90b497e881e4b9e8433b308057645bf` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:24.974870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77536cf3-2ce2-4112-b2d8-bfd3311be1cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f90b497e881e4b9e8433b308057645bf)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:24.974870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77536cf3-2ce2-4112-b2d8-bfd3311be1cc admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:25.025630 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-77536cf3-2ce2-4112-b2d8-bfd3311be1cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f90b497e881e4b9e8433b308057645bf', 'request_id': 'req-77536cf3-2ce2-4112-b2d8-bfd3311be1cc'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:25.034731 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-77536cf3-2ce2-4112-b2d8-bfd3311be1cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f90b497e881e4b9e8433b308057645bf', 'request_id': 'req-77536cf3-2ce2-4112-b2d8-bfd3311be1cc'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:25.043594 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-77536cf3-2ce2-4112-b2d8-bfd3311be1cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f90b497e881e4b9e8433b308057645bf', 'request_id': 'req-77536cf3-2ce2-4112-b2d8-bfd3311be1cc'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:25.056223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-77536cf3-2ce2-4112-b2d8-bfd3311be1cc admin admin] The token cache is being invalidate because project f90b497e881e4b9e8433b308057645bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:21:25.056549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-77536cf3-2ce2-4112-b2d8-bfd3311be1cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:25.057189 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-77536cf3-2ce2-4112-b2d8-bfd3311be1cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:25.058700 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 974/1954] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:21:24 2022] DELETE /identity/v3/projects/f90b497e881e4b9e8433b308057645bf => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:25.062820 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5393c-f948-42d8-877a-7289b5b92d8f service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:25.074767 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5393c-f948-42d8-877a-7289b5b92d8f service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:25.107661 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:25.138493 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f233b-f021-40c5-adf0-d0fbaaad5c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:25.139303 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f233b-f021-40c5-adf0-d0fbaaad5c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:25.139639 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f233b-f021-40c5-adf0-d0fbaaad5c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:25.140004 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f233b-f021-40c5-adf0-d0fbaaad5c7e admin admin] PATH_INFO: `/v3/users/ef1daf57da074905bed6497ce0c67887` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:25.141207 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f233b-f021-40c5-adf0-d0fbaaad5c7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef1daf57da074905bed6497ce0c67887)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:25.143262 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f233b-f021-40c5-adf0-d0fbaaad5c7e admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:25.196788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d92f233b-f021-40c5-adf0-d0fbaaad5c7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef1daf57da074905bed6497ce0c67887'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:25.205968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d92f233b-f021-40c5-adf0-d0fbaaad5c7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef1daf57da074905bed6497ce0c67887'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:25.217592 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d92f233b-f021-40c5-adf0-d0fbaaad5c7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef1daf57da074905bed6497ce0c67887'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:25.218500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d92f233b-f021-40c5-adf0-d0fbaaad5c7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef1daf57da074905bed6497ce0c67887'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:25.219163 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d92f233b-f021-40c5-adf0-d0fbaaad5c7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef1daf57da074905bed6497ce0c67887'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:25.244773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 981/1955] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:21:25 2022] DELETE /identity/v3/users/ef1daf57da074905bed6497ce0c67887 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:25.562555 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77536cf3-2ce2-4112-b2d8-bfd3311be1cc admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:25.566396 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-77536cf3-2ce2-4112-b2d8-bfd3311be1cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:25.578033 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77536cf3-2ce2-4112-b2d8-bfd3311be1cc admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:25.647727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa0f6e-e503-423e-8000-13cf7571b5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:25.648314 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa0f6e-e503-423e-8000-13cf7571b5b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:25.648598 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa0f6e-e503-423e-8000-13cf7571b5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:25.649062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa0f6e-e503-423e-8000-13cf7571b5b0 admin admin] PATH_INFO: `/v3/projects/90daa297b5384564bcdae8c92acc9672` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:25.650210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa0f6e-e503-423e-8000-13cf7571b5b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90daa297b5384564bcdae8c92acc9672)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:25.652437 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa0f6e-e503-423e-8000-13cf7571b5b0 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:25.706016 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9baa0f6e-e503-423e-8000-13cf7571b5b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90daa297b5384564bcdae8c92acc9672', 'request_id': 'req-9baa0f6e-e503-423e-8000-13cf7571b5b0'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:25.717616 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9baa0f6e-e503-423e-8000-13cf7571b5b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90daa297b5384564bcdae8c92acc9672', 'request_id': 'req-9baa0f6e-e503-423e-8000-13cf7571b5b0'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:25.723945 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9baa0f6e-e503-423e-8000-13cf7571b5b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90daa297b5384564bcdae8c92acc9672', 'request_id': 'req-9baa0f6e-e503-423e-8000-13cf7571b5b0'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:25.725250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:25.739539 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9baa0f6e-e503-423e-8000-13cf7571b5b0 admin admin] The token cache is being invalidate because project 90daa297b5384564bcdae8c92acc9672 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:21:25.740080 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9baa0f6e-e503-423e-8000-13cf7571b5b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:25.741059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9baa0f6e-e503-423e-8000-13cf7571b5b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:25.743200 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 975/1956] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:21:25 2022] DELETE /identity/v3/projects/90daa297b5384564bcdae8c92acc9672 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:25.748299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27fe06-c181-4619-ac09-c2e76d8b2eeb None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:25.748588 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27fe06-c181-4619-ac09-c2e76d8b2eeb None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:25.748870 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27fe06-c181-4619-ac09-c2e76d8b2eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:25.769055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-2c27fe06-c181-4619-ac09-c2e76d8b2eeb None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:25.770764 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-2c27fe06-c181-4619-ac09-c2e76d8b2eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:25.869148 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 982/1957] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:25.876564 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed679099-4b02-47ba-ad13-10afecf972f6 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:25.876852 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed679099-4b02-47ba-ad13-10afecf972f6 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:25.877130 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed679099-4b02-47ba-ad13-10afecf972f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:25.898278 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-ed679099-4b02-47ba-ad13-10afecf972f6 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:25.899584 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-ed679099-4b02-47ba-ad13-10afecf972f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:25.911505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 976/1958] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:25.915807 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27fe06-c181-4619-ac09-c2e76d8b2eeb None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:25.930782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27fe06-c181-4619-ac09-c2e76d8b2eeb None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:25.937148 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f323818-b139-4966-9e04-58a459f8a1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:25.937938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f323818-b139-4966-9e04-58a459f8a1f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:25.938532 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f323818-b139-4966-9e04-58a459f8a1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:25.938808 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f323818-b139-4966-9e04-58a459f8a1f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:25.939521 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f323818-b139-4966-9e04-58a459f8a1f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:21:25.939847 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f323818-b139-4966-9e04-58a459f8a1f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:25.941693 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f323818-b139-4966-9e04-58a459f8a1f0 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:25.966550 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 983/1959] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:21:25.973900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed679099-4b02-47ba-ad13-10afecf972f6 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:25.994616 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed679099-4b02-47ba-ad13-10afecf972f6 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:26.009620 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b0472-3b44-481e-9c82-c2b42c376efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:26.010559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b0472-3b44-481e-9c82-c2b42c376efd admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:26.010946 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b0472-3b44-481e-9c82-c2b42c376efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:26.011635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b0472-3b44-481e-9c82-c2b42c376efd admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:26.012626 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b0472-3b44-481e-9c82-c2b42c376efd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:26.015702 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b0472-3b44-481e-9c82-c2b42c376efd admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:26.077707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 977/1960] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:21:25 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:26.081684 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f323818-b139-4966-9e04-58a459f8a1f0 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:26.096626 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f323818-b139-4966-9e04-58a459f8a1f0 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:26.139424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:26.160096 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902901f-639b-4eb1-ab02-8cabfef59b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:26.160756 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902901f-639b-4eb1-ab02-8cabfef59b19 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:26.161034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902901f-639b-4eb1-ab02-8cabfef59b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:26.161315 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902901f-639b-4eb1-ab02-8cabfef59b19 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:26.162003 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b902901f-639b-4eb1-ab02-8cabfef59b19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:26.163940 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b902901f-639b-4eb1-ab02-8cabfef59b19 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:26.185881 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 984/1961] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:21:26 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:26.190160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b0472-3b44-481e-9c82-c2b42c376efd admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:26.201278 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b0472-3b44-481e-9c82-c2b42c376efd admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:26.205896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53793a-683a-42c9-8a14-b1835a981f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:26.206494 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53793a-683a-42c9-8a14-b1835a981f20 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:26.206774 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53793a-683a-42c9-8a14-b1835a981f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:26.207058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53793a-683a-42c9-8a14-b1835a981f20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:26.207722 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53793a-683a-42c9-8a14-b1835a981f20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:26.209307 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53793a-683a-42c9-8a14-b1835a981f20 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:26.219552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 978/1962] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:26.223471 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902901f-639b-4eb1-ab02-8cabfef59b19 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:26.234810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902901f-639b-4eb1-ab02-8cabfef59b19 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:26.239548 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa2755-ffcc-4b07-8734-4a7688357328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:26.240158 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa2755-ffcc-4b07-8734-4a7688357328 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:26.240458 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa2755-ffcc-4b07-8734-4a7688357328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:26.240724 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa2755-ffcc-4b07-8734-4a7688357328 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:26.241374 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efa2755-ffcc-4b07-8734-4a7688357328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:26.243773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efa2755-ffcc-4b07-8734-4a7688357328 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:26.254695 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 985/1963] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:26.259520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53793a-683a-42c9-8a14-b1835a981f20 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:26.269985 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53793a-683a-42c9-8a14-b1835a981f20 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:26.274851 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414279c2-3117-4460-aeed-6f76a5c73887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:26.275540 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414279c2-3117-4460-aeed-6f76a5c73887 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:26.275819 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414279c2-3117-4460-aeed-6f76a5c73887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:26.276087 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414279c2-3117-4460-aeed-6f76a5c73887 admin admin] PATH_INFO: `/v3/projects/d94c6a3ef756488eac765456d78ff335/users/95e5a1b739a54c3aa4dcb53e5bbe0452/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:26.292918 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414279c2-3117-4460-aeed-6f76a5c73887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d94c6a3ef756488eac765456d78ff335, user_id=95e5a1b739a54c3aa4dcb53e5bbe0452, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:26.296239 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414279c2-3117-4460-aeed-6f76a5c73887 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:26.315053 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 979/1964] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:26 2022] PUT /identity/v3/projects/d94c6a3ef756488eac765456d78ff335/users/95e5a1b739a54c3aa4dcb53e5bbe0452/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:26.319951 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011df71c-f602-4555-81ab-a1bdece7046e None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:26.320906 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011df71c-f602-4555-81ab-a1bdece7046e None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:26.321233 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011df71c-f602-4555-81ab-a1bdece7046e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:26.343507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-011df71c-f602-4555-81ab-a1bdece7046e None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:26.344751 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-011df71c-f602-4555-81ab-a1bdece7046e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:26.386301 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414279c2-3117-4460-aeed-6f76a5c73887 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:26.388384 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-414279c2-3117-4460-aeed-6f76a5c73887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:26.421355 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 986/1965] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:26.432715 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011df71c-f602-4555-81ab-a1bdece7046e None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:26.435155 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-011df71c-f602-4555-81ab-a1bdece7046e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:26.460877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414279c2-3117-4460-aeed-6f76a5c73887 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:26.465882 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7292cc-c141-4c24-b07c-4b83b9be9ffe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:26.466651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7292cc-c141-4c24-b07c-4b83b9be9ffe service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:26.466941 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7292cc-c141-4c24-b07c-4b83b9be9ffe service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:26.467298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7292cc-c141-4c24-b07c-4b83b9be9ffe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:26.470730 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-2f7292cc-c141-4c24-b07c-4b83b9be9ffe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:26.507852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011df71c-f602-4555-81ab-a1bdece7046e None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:26.512227 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b8f6d-a8b0-4f11-b07a-aaa78e08a535 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:26.512897 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b8f6d-a8b0-4f11-b07a-aaa78e08a535 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:26.513470 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b8f6d-a8b0-4f11-b07a-aaa78e08a535 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:26.513470 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b8f6d-a8b0-4f11-b07a-aaa78e08a535 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:26.525687 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b8f6d-a8b0-4f11-b07a-aaa78e08a535 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:26.527760 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b8f6d-a8b0-4f11-b07a-aaa78e08a535 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:26.540198 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 987/1966] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:21:26.552207 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7292cc-c141-4c24-b07c-4b83b9be9ffe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:26.553681 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7292cc-c141-4c24-b07c-4b83b9be9ffe service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:26.564353 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 980/1967] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:26 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:21:27.381631 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce69366-3b33-4618-8a1d-b7e16d7a0afe None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:27.382162 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce69366-3b33-4618-8a1d-b7e16d7a0afe None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:27.382251 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce69366-3b33-4618-8a1d-b7e16d7a0afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:27.421050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-dce69366-3b33-4618-8a1d-b7e16d7a0afe None None] MFA Rules not processed for user `95e5a1b739a54c3aa4dcb53e5bbe0452`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:27.422429 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-dce69366-3b33-4618-8a1d-b7e16d7a0afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:27.498837 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:27.519320 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 988/1968] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:27.530626 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7292cc-c141-4c24-b07c-4b83b9be9ffe service cinder] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:27.532174 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-2f7292cc-c141-4c24-b07c-4b83b9be9ffe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:27.544346 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7292cc-c141-4c24-b07c-4b83b9be9ffe service cinder] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:27.547637 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b2a854-3b6a-4277-80d8-c7f1c3f121c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:27.548254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2a854-3b6a-4277-80d8-c7f1c3f121c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:27.548532 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2a854-3b6a-4277-80d8-c7f1c3f121c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:27.548802 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2a854-3b6a-4277-80d8-c7f1c3f121c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:27.568041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b2a854-3b6a-4277-80d8-c7f1c3f121c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:27.569559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b2a854-3b6a-4277-80d8-c7f1c3f121c1 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:27.581865 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 981/1969] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:21:28.285814 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce69366-3b33-4618-8a1d-b7e16d7a0afe None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:28.295906 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce69366-3b33-4618-8a1d-b7e16d7a0afe None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:28.299466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69165bf1-2c7c-494b-a65c-e3fd702ed03b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:28.300068 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69165bf1-2c7c-494b-a65c-e3fd702ed03b service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:28.300360 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69165bf1-2c7c-494b-a65c-e3fd702ed03b service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:28.300633 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69165bf1-2c7c-494b-a65c-e3fd702ed03b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:28.302401 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-69165bf1-2c7c-494b-a65c-e3fd702ed03b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:28.372920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69165bf1-2c7c-494b-a65c-e3fd702ed03b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:28.374475 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69165bf1-2c7c-494b-a65c-e3fd702ed03b service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:28.388206 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 989/1970] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:28 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:21:30.102249 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0f82a-fd78-4acb-b914-93f52a600278 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:30.102839 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0f82a-fd78-4acb-b914-93f52a600278 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:30.102839 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0f82a-fd78-4acb-b914-93f52a600278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:30.135517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-9ca0f82a-fd78-4acb-b914-93f52a600278 None None] MFA Rules not processed for user `95e5a1b739a54c3aa4dcb53e5bbe0452`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:30.137010 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-9ca0f82a-fd78-4acb-b914-93f52a600278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:30.144371 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 982/1971] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:38.382078 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=34 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:40.054260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69165bf1-2c7c-494b-a65c-e3fd702ed03b service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:40.056501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-69165bf1-2c7c-494b-a65c-e3fd702ed03b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:40.075277 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69165bf1-2c7c-494b-a65c-e3fd702ed03b service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:40.077675 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210bff5a-96f1-4ebc-82f3-ca22d3a4e623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:40.078606 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210bff5a-96f1-4ebc-82f3-ca22d3a4e623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:40.079105 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210bff5a-96f1-4ebc-82f3-ca22d3a4e623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:40.079767 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210bff5a-96f1-4ebc-82f3-ca22d3a4e623 admin admin] PATH_INFO: `/v3/users/327500a37ab44a489329f849865e996f` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:40.081203 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210bff5a-96f1-4ebc-82f3-ca22d3a4e623 admin admin] RBAC: Authorizing `identity:delete_user(user_id=327500a37ab44a489329f849865e996f)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:40.083465 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210bff5a-96f1-4ebc-82f3-ca22d3a4e623 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:40.125559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=89,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:40.154903 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-210bff5a-96f1-4ebc-82f3-ca22d3a4e623 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '327500a37ab44a489329f849865e996f'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:40.163399 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-210bff5a-96f1-4ebc-82f3-ca22d3a4e623 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '327500a37ab44a489329f849865e996f'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:40.171613 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-210bff5a-96f1-4ebc-82f3-ca22d3a4e623 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '327500a37ab44a489329f849865e996f'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:40.172384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-210bff5a-96f1-4ebc-82f3-ca22d3a4e623 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '327500a37ab44a489329f849865e996f'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:40.173085 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-210bff5a-96f1-4ebc-82f3-ca22d3a4e623 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '327500a37ab44a489329f849865e996f'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:40.192132 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 990/1972] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:21:40 2022] DELETE /identity/v3/users/327500a37ab44a489329f849865e996f => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:40.196530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158afc82-9598-4464-9aa1-a2bdac0a7de0 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:40.196809 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158afc82-9598-4464-9aa1-a2bdac0a7de0 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:40.197106 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158afc82-9598-4464-9aa1-a2bdac0a7de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:40.215484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-158afc82-9598-4464-9aa1-a2bdac0a7de0 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:40.216663 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-158afc82-9598-4464-9aa1-a2bdac0a7de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:40.275654 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 983/1973] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:40.285335 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210bff5a-96f1-4ebc-82f3-ca22d3a4e623 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:40.286720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-210bff5a-96f1-4ebc-82f3-ca22d3a4e623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:40.342706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210bff5a-96f1-4ebc-82f3-ca22d3a4e623 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:40.346401 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb75c27-d0f6-45e3-a78c-a6109c4f2e7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:40.347060 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb75c27-d0f6-45e3-a78c-a6109c4f2e7b service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:40.347353 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb75c27-d0f6-45e3-a78c-a6109c4f2e7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:40.347628 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb75c27-d0f6-45e3-a78c-a6109c4f2e7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:40.358003 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb75c27-d0f6-45e3-a78c-a6109c4f2e7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:40.359480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb75c27-d0f6-45e3-a78c-a6109c4f2e7b service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:40.370275 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 991/1974] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:21:40.650637 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158afc82-9598-4464-9aa1-a2bdac0a7de0 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:40.652426 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-158afc82-9598-4464-9aa1-a2bdac0a7de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:40.675074 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158afc82-9598-4464-9aa1-a2bdac0a7de0 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:40.681711 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d85924-8aea-4940-a984-acdad79aeb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:40.682545 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d85924-8aea-4940-a984-acdad79aeb86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:40.682957 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d85924-8aea-4940-a984-acdad79aeb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:40.683419 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d85924-8aea-4940-a984-acdad79aeb86 admin admin] PATH_INFO: `/v3/projects/1c97c362165a4e2dbed71989b2b37158` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:40.684920 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d85924-8aea-4940-a984-acdad79aeb86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c97c362165a4e2dbed71989b2b37158)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:40.687439 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d85924-8aea-4940-a984-acdad79aeb86 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:40.751143 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-34d85924-8aea-4940-a984-acdad79aeb86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c97c362165a4e2dbed71989b2b37158', 'request_id': 'req-34d85924-8aea-4940-a984-acdad79aeb86'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:40.779290 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-34d85924-8aea-4940-a984-acdad79aeb86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c97c362165a4e2dbed71989b2b37158', 'request_id': 'req-34d85924-8aea-4940-a984-acdad79aeb86'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:40.786585 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-34d85924-8aea-4940-a984-acdad79aeb86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c97c362165a4e2dbed71989b2b37158', 'request_id': 'req-34d85924-8aea-4940-a984-acdad79aeb86'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:40.803991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-34d85924-8aea-4940-a984-acdad79aeb86 admin admin] The token cache is being invalidate because project 1c97c362165a4e2dbed71989b2b37158 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:21:40.804362 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-34d85924-8aea-4940-a984-acdad79aeb86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:40.810751 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-34d85924-8aea-4940-a984-acdad79aeb86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:40.814895 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 984/1975] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:21:40 2022] DELETE /identity/v3/projects/1c97c362165a4e2dbed71989b2b37158 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:40.818779 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb75c27-d0f6-45e3-a78c-a6109c4f2e7b service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:40.831774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb75c27-d0f6-45e3-a78c-a6109c4f2e7b service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:40.925004 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec35f8-1fce-4ea6-9a7c-b09942d763b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:40.925668 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec35f8-1fce-4ea6-9a7c-b09942d763b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:40.925952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec35f8-1fce-4ea6-9a7c-b09942d763b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:40.926227 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec35f8-1fce-4ea6-9a7c-b09942d763b6 admin admin] PATH_INFO: `/v3/users/f74210bbd3de418ca42e8df90c6fd78a` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:40.927731 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec35f8-1fce-4ea6-9a7c-b09942d763b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f74210bbd3de418ca42e8df90c6fd78a)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:40.930365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec35f8-1fce-4ea6-9a7c-b09942d763b6 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:40.968553 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:41.024817 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-93ec35f8-1fce-4ea6-9a7c-b09942d763b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f74210bbd3de418ca42e8df90c6fd78a'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:41.034797 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-93ec35f8-1fce-4ea6-9a7c-b09942d763b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f74210bbd3de418ca42e8df90c6fd78a'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:41.045217 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-93ec35f8-1fce-4ea6-9a7c-b09942d763b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f74210bbd3de418ca42e8df90c6fd78a'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:41.046097 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-93ec35f8-1fce-4ea6-9a7c-b09942d763b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f74210bbd3de418ca42e8df90c6fd78a'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:41.046707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-93ec35f8-1fce-4ea6-9a7c-b09942d763b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f74210bbd3de418ca42e8df90c6fd78a'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:41.085432 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 992/1976] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:21:40 2022] DELETE /identity/v3/users/f74210bbd3de418ca42e8df90c6fd78a => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:41.407725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d85924-8aea-4940-a984-acdad79aeb86 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:41.409693 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-34d85924-8aea-4940-a984-acdad79aeb86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:41.435324 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d85924-8aea-4940-a984-acdad79aeb86 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:41.505700 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b79eb3-5fbe-4964-ad07-0e43a09b822c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:41.506350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b79eb3-5fbe-4964-ad07-0e43a09b822c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:41.506633 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b79eb3-5fbe-4964-ad07-0e43a09b822c admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:41.506907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b79eb3-5fbe-4964-ad07-0e43a09b822c admin admin] PATH_INFO: `/v3/projects/72bae66e0ef44e66a993a7a880489244` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:41.508441 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b79eb3-5fbe-4964-ad07-0e43a09b822c admin admin] RBAC: Authorizing `identity:delete_project(project_id=72bae66e0ef44e66a993a7a880489244)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:41.510487 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b79eb3-5fbe-4964-ad07-0e43a09b822c admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:41.542578 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:41.560876 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-75b79eb3-5fbe-4964-ad07-0e43a09b822c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72bae66e0ef44e66a993a7a880489244', 'request_id': 'req-75b79eb3-5fbe-4964-ad07-0e43a09b822c'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:41.568145 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-75b79eb3-5fbe-4964-ad07-0e43a09b822c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72bae66e0ef44e66a993a7a880489244', 'request_id': 'req-75b79eb3-5fbe-4964-ad07-0e43a09b822c'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:41.574314 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-75b79eb3-5fbe-4964-ad07-0e43a09b822c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72bae66e0ef44e66a993a7a880489244', 'request_id': 'req-75b79eb3-5fbe-4964-ad07-0e43a09b822c'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:41.585789 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-75b79eb3-5fbe-4964-ad07-0e43a09b822c admin admin] The token cache is being invalidate because project 72bae66e0ef44e66a993a7a880489244 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:21:41.586125 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-75b79eb3-5fbe-4964-ad07-0e43a09b822c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:41.587043 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-75b79eb3-5fbe-4964-ad07-0e43a09b822c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:41.588648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 985/1977] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:21:41 2022] DELETE /identity/v3/projects/72bae66e0ef44e66a993a7a880489244 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:41.595347 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e15fa3-8bb2-4052-9c1d-0fd5366fb90c None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:41.595668 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e15fa3-8bb2-4052-9c1d-0fd5366fb90c None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:41.595949 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e15fa3-8bb2-4052-9c1d-0fd5366fb90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:41.617627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-80e15fa3-8bb2-4052-9c1d-0fd5366fb90c None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:41.619314 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-80e15fa3-8bb2-4052-9c1d-0fd5366fb90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:41.683865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 993/1978] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:41.692333 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567ba32-8f38-4277-a9f9-d429951785cc None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:41.692626 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567ba32-8f38-4277-a9f9-d429951785cc None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:41.692900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567ba32-8f38-4277-a9f9-d429951785cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:41.714035 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-3567ba32-8f38-4277-a9f9-d429951785cc None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:41.715308 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-3567ba32-8f38-4277-a9f9-d429951785cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:41.724854 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 986/1979] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:21:41.730152 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e15fa3-8bb2-4052-9c1d-0fd5366fb90c None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:41.741006 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e15fa3-8bb2-4052-9c1d-0fd5366fb90c None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:41.746998 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afdd248-8760-445a-a682-c62c995b7088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:41.747658 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdd248-8760-445a-a682-c62c995b7088 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:41.747944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdd248-8760-445a-a682-c62c995b7088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:41.748200 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdd248-8760-445a-a682-c62c995b7088 admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:41.748852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afdd248-8760-445a-a682-c62c995b7088 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:21:41.749151 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afdd248-8760-445a-a682-c62c995b7088 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:41.750799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afdd248-8760-445a-a682-c62c995b7088 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:41.767653 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 994/1980] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:21:41.771670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3567ba32-8f38-4277-a9f9-d429951785cc None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:41.786178 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3567ba32-8f38-4277-a9f9-d429951785cc None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:41.792138 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6376885-fa87-40c8-9efd-d561f974dfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:41.793009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6376885-fa87-40c8-9efd-d561f974dfdd admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:41.793653 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6376885-fa87-40c8-9efd-d561f974dfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:41.793653 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6376885-fa87-40c8-9efd-d561f974dfdd admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:41.794367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6376885-fa87-40c8-9efd-d561f974dfdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:41.796366 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6376885-fa87-40c8-9efd-d561f974dfdd admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:41.834235 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 987/1981] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:21:41 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:41.835360 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afdd248-8760-445a-a682-c62c995b7088 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:41.845738 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afdd248-8760-445a-a682-c62c995b7088 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:41.912644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb7bb0-f945-4903-9856-1a34d686da94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:41.913320 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb7bb0-f945-4903-9856-1a34d686da94 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:41.913600 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb7bb0-f945-4903-9856-1a34d686da94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:41.913925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb7bb0-f945-4903-9856-1a34d686da94 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:41.915256 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb7bb0-f945-4903-9856-1a34d686da94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:41.917572 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb7bb0-f945-4903-9856-1a34d686da94 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:41.940678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 995/1982] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:21:41 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:21:41.944827 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6376885-fa87-40c8-9efd-d561f974dfdd admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:41.956335 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6376885-fa87-40c8-9efd-d561f974dfdd admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:41.961649 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c6743-4c03-4f5a-9a15-4a5298e1bab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:41.962288 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c6743-4c03-4f5a-9a15-4a5298e1bab4 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:41.962658 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c6743-4c03-4f5a-9a15-4a5298e1bab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:41.962935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c6743-4c03-4f5a-9a15-4a5298e1bab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:41.963624 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627c6743-4c03-4f5a-9a15-4a5298e1bab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:41.965218 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627c6743-4c03-4f5a-9a15-4a5298e1bab4 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:41.979558 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 988/1983] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:41.980642 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb7bb0-f945-4903-9856-1a34d686da94 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:41.990621 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb7bb0-f945-4903-9856-1a34d686da94 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:41.996370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61432e8e-4578-41a8-9678-4143874e9908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:41.997375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61432e8e-4578-41a8-9678-4143874e9908 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:41.997662 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61432e8e-4578-41a8-9678-4143874e9908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:41.997937 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61432e8e-4578-41a8-9678-4143874e9908 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:41.998642 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61432e8e-4578-41a8-9678-4143874e9908 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:42.000223 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61432e8e-4578-41a8-9678-4143874e9908 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:42.005855 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:42.015928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 996/1984] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:42.017262 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c6743-4c03-4f5a-9a15-4a5298e1bab4 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:42.028625 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c6743-4c03-4f5a-9a15-4a5298e1bab4 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:42.033498 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af74b7-dba3-4fc3-9fb9-3f8ea5960cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:42.034159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af74b7-dba3-4fc3-9fb9-3f8ea5960cc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:42.034439 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af74b7-dba3-4fc3-9fb9-3f8ea5960cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:42.034740 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af74b7-dba3-4fc3-9fb9-3f8ea5960cc8 admin admin] PATH_INFO: `/v3/projects/26c75fe8d9b64910b0f238cacade3d2d/users/4e2ab57d80f144fe8261141bd7c214b0/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:42.050953 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af74b7-dba3-4fc3-9fb9-3f8ea5960cc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26c75fe8d9b64910b0f238cacade3d2d, user_id=4e2ab57d80f144fe8261141bd7c214b0, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:42.053723 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af74b7-dba3-4fc3-9fb9-3f8ea5960cc8 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:42.068932 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 989/1985] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:42 2022] PUT /identity/v3/projects/26c75fe8d9b64910b0f238cacade3d2d/users/4e2ab57d80f144fe8261141bd7c214b0/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:42.081648 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff45d4b-58ad-4e98-944d-17570ffc0321 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:42.081940 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff45d4b-58ad-4e98-944d-17570ffc0321 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:42.082218 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff45d4b-58ad-4e98-944d-17570ffc0321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:42.115515 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-fff45d4b-58ad-4e98-944d-17570ffc0321 None None] MFA Rules not processed for user `4e2ab57d80f144fe8261141bd7c214b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:42.116904 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-fff45d4b-58ad-4e98-944d-17570ffc0321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:42.187516 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 997/1986] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:21:42.199948 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af74b7-dba3-4fc3-9fb9-3f8ea5960cc8 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:42.201620 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-20af74b7-dba3-4fc3-9fb9-3f8ea5960cc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:42.263124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af74b7-dba3-4fc3-9fb9-3f8ea5960cc8 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:42.267718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6e621-8997-449b-9805-007eac83bb54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:42.268333 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6e621-8997-449b-9805-007eac83bb54 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:42.268609 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6e621-8997-449b-9805-007eac83bb54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:42.268875 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6e621-8997-449b-9805-007eac83bb54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:42.278385 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee6e621-8997-449b-9805-007eac83bb54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:42.279839 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee6e621-8997-449b-9805-007eac83bb54 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:42.288982 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 990/1987] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:21:42.964120 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff45d4b-58ad-4e98-944d-17570ffc0321 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:42.965970 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-fff45d4b-58ad-4e98-944d-17570ffc0321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:43.043019 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff45d4b-58ad-4e98-944d-17570ffc0321 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:43.048340 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ea9c4-db8a-4eec-97dc-fa764693a01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:43.048978 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ea9c4-db8a-4eec-97dc-fa764693a01d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:43.049260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ea9c4-db8a-4eec-97dc-fa764693a01d admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:43.049537 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ea9c4-db8a-4eec-97dc-fa764693a01d admin admin] PATH_INFO: `/v3/users/bd8f211823cd41d9a826e56a10e33e11` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:43.050668 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898ea9c4-db8a-4eec-97dc-fa764693a01d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd8f211823cd41d9a826e56a10e33e11)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:43.052659 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898ea9c4-db8a-4eec-97dc-fa764693a01d admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:43.109514 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-898ea9c4-db8a-4eec-97dc-fa764693a01d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd8f211823cd41d9a826e56a10e33e11'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:43.119580 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-898ea9c4-db8a-4eec-97dc-fa764693a01d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd8f211823cd41d9a826e56a10e33e11'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:43.127231 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-898ea9c4-db8a-4eec-97dc-fa764693a01d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd8f211823cd41d9a826e56a10e33e11'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:43.127922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-898ea9c4-db8a-4eec-97dc-fa764693a01d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd8f211823cd41d9a826e56a10e33e11'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:43.128830 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-898ea9c4-db8a-4eec-97dc-fa764693a01d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd8f211823cd41d9a826e56a10e33e11'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:43.146898 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 998/1988] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:21:42 2022] DELETE /identity/v3/users/bd8f211823cd41d9a826e56a10e33e11 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:43.151560 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a753480-d825-4961-889f-8482b535172b None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:43.152056 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a753480-d825-4961-889f-8482b535172b None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:43.152156 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a753480-d825-4961-889f-8482b535172b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:43.173216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-3a753480-d825-4961-889f-8482b535172b None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:43.174428 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-3a753480-d825-4961-889f-8482b535172b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:43.224022 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:43.238949 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 991/1989] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:43.249559 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ea9c4-db8a-4eec-97dc-fa764693a01d admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:43.251195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-898ea9c4-db8a-4eec-97dc-fa764693a01d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:43.263191 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:43.320743 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ea9c4-db8a-4eec-97dc-fa764693a01d admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:43.324415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fc177-e86d-4062-8912-2c579ed4695e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:43.325048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fc177-e86d-4062-8912-2c579ed4695e service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:43.325327 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fc177-e86d-4062-8912-2c579ed4695e service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:43.325597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fc177-e86d-4062-8912-2c579ed4695e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:43.334898 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fc177-e86d-4062-8912-2c579ed4695e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:43.336487 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fc177-e86d-4062-8912-2c579ed4695e service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:43.346209 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 999/1990] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:21:43.677335 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a753480-d825-4961-889f-8482b535172b None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:43.678828 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-3a753480-d825-4961-889f-8482b535172b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:43.692856 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a753480-d825-4961-889f-8482b535172b None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:43.698502 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114da3b-27d0-4266-a9fe-5d941f3d4202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:43.699355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114da3b-27d0-4266-a9fe-5d941f3d4202 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:43.699684 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114da3b-27d0-4266-a9fe-5d941f3d4202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:43.700034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114da3b-27d0-4266-a9fe-5d941f3d4202 admin admin] PATH_INFO: `/v3/projects/5b50fab50ec442f08abb008f401b35f4` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:43.701341 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114da3b-27d0-4266-a9fe-5d941f3d4202 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b50fab50ec442f08abb008f401b35f4)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:43.704123 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114da3b-27d0-4266-a9fe-5d941f3d4202 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:43.763421 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b114da3b-27d0-4266-a9fe-5d941f3d4202 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b50fab50ec442f08abb008f401b35f4', 'request_id': 'req-b114da3b-27d0-4266-a9fe-5d941f3d4202'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:43.787521 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b114da3b-27d0-4266-a9fe-5d941f3d4202 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b50fab50ec442f08abb008f401b35f4', 'request_id': 'req-b114da3b-27d0-4266-a9fe-5d941f3d4202'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:43.795184 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b114da3b-27d0-4266-a9fe-5d941f3d4202 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b50fab50ec442f08abb008f401b35f4', 'request_id': 'req-b114da3b-27d0-4266-a9fe-5d941f3d4202'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:43.808793 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b114da3b-27d0-4266-a9fe-5d941f3d4202 admin admin] The token cache is being invalidate because project 5b50fab50ec442f08abb008f401b35f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:21:43.809123 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b114da3b-27d0-4266-a9fe-5d941f3d4202 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:43.809879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b114da3b-27d0-4266-a9fe-5d941f3d4202 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:43.811742 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 992/1991] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:21:43 2022] DELETE /identity/v3/projects/5b50fab50ec442f08abb008f401b35f4 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:43.815698 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fc177-e86d-4062-8912-2c579ed4695e service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:43.827267 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fc177-e86d-4062-8912-2c579ed4695e service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:43.891328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd53188-4b8f-471c-bd80-d519078da7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:43.892077 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd53188-4b8f-471c-bd80-d519078da7fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:43.892380 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd53188-4b8f-471c-bd80-d519078da7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:43.892658 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd53188-4b8f-471c-bd80-d519078da7fc admin admin] PATH_INFO: `/v3/users/367150656f694352bbc0ffa8aba6a5eb` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:43.893792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd53188-4b8f-471c-bd80-d519078da7fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=367150656f694352bbc0ffa8aba6a5eb)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:43.895786 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd53188-4b8f-471c-bd80-d519078da7fc admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:43.963383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-9cd53188-4b8f-471c-bd80-d519078da7fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '367150656f694352bbc0ffa8aba6a5eb'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:43.976452 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-9cd53188-4b8f-471c-bd80-d519078da7fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '367150656f694352bbc0ffa8aba6a5eb'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:43.985208 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-9cd53188-4b8f-471c-bd80-d519078da7fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '367150656f694352bbc0ffa8aba6a5eb'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:43.985842 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-9cd53188-4b8f-471c-bd80-d519078da7fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '367150656f694352bbc0ffa8aba6a5eb'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:43.986473 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-9cd53188-4b8f-471c-bd80-d519078da7fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '367150656f694352bbc0ffa8aba6a5eb'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:44.007601 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1000/1992] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:21:43 2022] DELETE /identity/v3/users/367150656f694352bbc0ffa8aba6a5eb => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:44.316603 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114da3b-27d0-4266-a9fe-5d941f3d4202 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:44.321473 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b114da3b-27d0-4266-a9fe-5d941f3d4202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:44.374351 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114da3b-27d0-4266-a9fe-5d941f3d4202 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:44.450627 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea1aa1-0811-4f95-b46c-c148eb1ab18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:44.451418 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea1aa1-0811-4f95-b46c-c148eb1ab18b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:44.451769 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea1aa1-0811-4f95-b46c-c148eb1ab18b admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:44.452113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea1aa1-0811-4f95-b46c-c148eb1ab18b admin admin] PATH_INFO: `/v3/projects/17bcc1d36e49400593dc98619efe7ac2` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:44.453315 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea1aa1-0811-4f95-b46c-c148eb1ab18b admin admin] RBAC: Authorizing `identity:delete_project(project_id=17bcc1d36e49400593dc98619efe7ac2)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:44.455349 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea1aa1-0811-4f95-b46c-c148eb1ab18b admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:44.506623 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-96ea1aa1-0811-4f95-b46c-c148eb1ab18b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17bcc1d36e49400593dc98619efe7ac2', 'request_id': 'req-96ea1aa1-0811-4f95-b46c-c148eb1ab18b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:44.515816 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-96ea1aa1-0811-4f95-b46c-c148eb1ab18b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17bcc1d36e49400593dc98619efe7ac2', 'request_id': 'req-96ea1aa1-0811-4f95-b46c-c148eb1ab18b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:44.521398 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-96ea1aa1-0811-4f95-b46c-c148eb1ab18b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17bcc1d36e49400593dc98619efe7ac2', 'request_id': 'req-96ea1aa1-0811-4f95-b46c-c148eb1ab18b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:44.533621 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-96ea1aa1-0811-4f95-b46c-c148eb1ab18b admin admin] The token cache is being invalidate because project 17bcc1d36e49400593dc98619efe7ac2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:21:44.533916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-96ea1aa1-0811-4f95-b46c-c148eb1ab18b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:44.534566 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-96ea1aa1-0811-4f95-b46c-c148eb1ab18b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:44.536112 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 993/1993] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:21:44 2022] DELETE /identity/v3/projects/17bcc1d36e49400593dc98619efe7ac2 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:44.541514 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26606e2a-d823-4034-806b-0d1139c6e79f None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:44.542027 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26606e2a-d823-4034-806b-0d1139c6e79f None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:44.542138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26606e2a-d823-4034-806b-0d1139c6e79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:44.561829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-26606e2a-d823-4034-806b-0d1139c6e79f None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:44.563062 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-26606e2a-d823-4034-806b-0d1139c6e79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:44.593574 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:44.640072 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1001/1994] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:44.647746 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a931abc-4ad6-4b4e-80cd-d976efee6356 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:44.648107 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a931abc-4ad6-4b4e-80cd-d976efee6356 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:44.648696 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a931abc-4ad6-4b4e-80cd-d976efee6356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:44.665000 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:44.671053 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-8a931abc-4ad6-4b4e-80cd-d976efee6356 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:44.673235 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-8a931abc-4ad6-4b4e-80cd-d976efee6356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:44.683024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 994/1995] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:44.687298 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26606e2a-d823-4034-806b-0d1139c6e79f None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:44.698235 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26606e2a-d823-4034-806b-0d1139c6e79f None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:44.711578 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e998c-2889-40aa-bde6-4bacd8c046a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:44.711578 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e998c-2889-40aa-bde6-4bacd8c046a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:44.711578 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e998c-2889-40aa-bde6-4bacd8c046a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:44.711578 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e998c-2889-40aa-bde6-4bacd8c046a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:44.711930 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226e998c-2889-40aa-bde6-4bacd8c046a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:21:44.711930 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226e998c-2889-40aa-bde6-4bacd8c046a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:44.711930 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226e998c-2889-40aa-bde6-4bacd8c046a9 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:44.723377 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1002/1996] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:21:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:21:44.728526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a931abc-4ad6-4b4e-80cd-d976efee6356 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:44.737712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a931abc-4ad6-4b4e-80cd-d976efee6356 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:44.742963 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b2446-52eb-45cf-8250-d9f55a4b903a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:44.745307 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b2446-52eb-45cf-8250-d9f55a4b903a admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:44.745588 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b2446-52eb-45cf-8250-d9f55a4b903a admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:44.745857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b2446-52eb-45cf-8250-d9f55a4b903a admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:44.746499 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b2446-52eb-45cf-8250-d9f55a4b903a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:44.749119 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b2446-52eb-45cf-8250-d9f55a4b903a admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:44.778867 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 995/1997] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:21:44 2022] POST /identity/v3/projects => generated 355 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:44.783544 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e998c-2889-40aa-bde6-4bacd8c046a9 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:44.795249 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e998c-2889-40aa-bde6-4bacd8c046a9 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:44.888571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca8c0b-f5f9-4682-8861-abf61cc4f191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:44.889532 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca8c0b-f5f9-4682-8861-abf61cc4f191 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:44.890026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca8c0b-f5f9-4682-8861-abf61cc4f191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:44.890641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca8c0b-f5f9-4682-8861-abf61cc4f191 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:44.891693 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca8c0b-f5f9-4682-8861-abf61cc4f191 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:44.894178 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca8c0b-f5f9-4682-8861-abf61cc4f191 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:44.939546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1003/1998] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:21:44 2022] POST /identity/v3/users => generated 278 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:44.940331 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b2446-52eb-45cf-8250-d9f55a4b903a admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:44.950921 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b2446-52eb-45cf-8250-d9f55a4b903a admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:44.956107 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43d214-7a5e-4d05-9e98-0aeca15b1856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:44.956724 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43d214-7a5e-4d05-9e98-0aeca15b1856 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:44.957003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43d214-7a5e-4d05-9e98-0aeca15b1856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:44.957274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43d214-7a5e-4d05-9e98-0aeca15b1856 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:44.957925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc43d214-7a5e-4d05-9e98-0aeca15b1856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:44.959618 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc43d214-7a5e-4d05-9e98-0aeca15b1856 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:44.980889 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 996/1999] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:21:44.984813 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca8c0b-f5f9-4682-8861-abf61cc4f191 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:45.000548 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca8c0b-f5f9-4682-8861-abf61cc4f191 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:45.007364 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4251a2-5574-442e-843b-73cd98147d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:45.008057 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4251a2-5574-442e-843b-73cd98147d66 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:45.008839 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4251a2-5574-442e-843b-73cd98147d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:45.009117 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4251a2-5574-442e-843b-73cd98147d66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:45.009776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4251a2-5574-442e-843b-73cd98147d66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:45.011484 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4251a2-5574-442e-843b-73cd98147d66 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:45.025437 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1004/2000] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:21:45.029508 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43d214-7a5e-4d05-9e98-0aeca15b1856 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:45.040093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43d214-7a5e-4d05-9e98-0aeca15b1856 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:45.045033 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396b2b1-4f92-4ebe-b160-4cf28f46c6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:45.045751 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396b2b1-4f92-4ebe-b160-4cf28f46c6c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:45.046042 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396b2b1-4f92-4ebe-b160-4cf28f46c6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:45.046380 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396b2b1-4f92-4ebe-b160-4cf28f46c6c5 admin admin] PATH_INFO: `/v3/projects/6b680d27f22e41bab009dc80a23d6496/users/04a20356b6cd44b8867b084ce0c02b32/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:45.075744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396b2b1-4f92-4ebe-b160-4cf28f46c6c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b680d27f22e41bab009dc80a23d6496, user_id=04a20356b6cd44b8867b084ce0c02b32, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:45.078639 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396b2b1-4f92-4ebe-b160-4cf28f46c6c5 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:45.095698 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 997/2001] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:45 2022] PUT /identity/v3/projects/6b680d27f22e41bab009dc80a23d6496/users/04a20356b6cd44b8867b084ce0c02b32/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:45.106596 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c71612-15eb-4403-bf73-1c9faa475459 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:45.107362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c71612-15eb-4403-bf73-1c9faa475459 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:45.107362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c71612-15eb-4403-bf73-1c9faa475459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:45.139385 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-09c71612-15eb-4403-bf73-1c9faa475459 None None] MFA Rules not processed for user `04a20356b6cd44b8867b084ce0c02b32`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:45.140602 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-09c71612-15eb-4403-bf73-1c9faa475459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:45.207808 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1005/2002] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:45 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:45.212924 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396b2b1-4f92-4ebe-b160-4cf28f46c6c5 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:45.282545 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396b2b1-4f92-4ebe-b160-4cf28f46c6c5 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:45.287279 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f359d-45fe-4571-af6c-decd50c8232c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:45.287890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f359d-45fe-4571-af6c-decd50c8232c admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:45.288265 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f359d-45fe-4571-af6c-decd50c8232c admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:45.288431 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f359d-45fe-4571-af6c-decd50c8232c admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:45.289570 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f359d-45fe-4571-af6c-decd50c8232c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:45.291872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f359d-45fe-4571-af6c-decd50c8232c admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:45.323669 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 998/2003] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:21:45 2022] POST /identity/v3/projects => generated 357 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:45.325070 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c71612-15eb-4403-bf73-1c9faa475459 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:45.339315 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c71612-15eb-4403-bf73-1c9faa475459 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:45.370344 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:45.418413 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80104e-cc66-4a1e-8e19-5fa0d4f40163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:45.419141 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80104e-cc66-4a1e-8e19-5fa0d4f40163 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:45.419507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80104e-cc66-4a1e-8e19-5fa0d4f40163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:45.419842 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80104e-cc66-4a1e-8e19-5fa0d4f40163 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:45.420633 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f80104e-cc66-4a1e-8e19-5fa0d4f40163 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:45.422569 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f80104e-cc66-4a1e-8e19-5fa0d4f40163 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:45.448282 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1006/2004] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:21:45 2022] POST /identity/v3/users => generated 285 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:45.452448 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f359d-45fe-4571-af6c-decd50c8232c admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:45.464113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f359d-45fe-4571-af6c-decd50c8232c admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:45.468850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9264cc3-87e4-4eb6-a08c-adb4c21d9e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:45.469465 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9264cc3-87e4-4eb6-a08c-adb4c21d9e31 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:45.469759 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9264cc3-87e4-4eb6-a08c-adb4c21d9e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:45.470164 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9264cc3-87e4-4eb6-a08c-adb4c21d9e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:45.471071 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9264cc3-87e4-4eb6-a08c-adb4c21d9e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:45.472720 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9264cc3-87e4-4eb6-a08c-adb4c21d9e31 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:45.487525 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 999/2005] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:45.491682 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80104e-cc66-4a1e-8e19-5fa0d4f40163 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:45.507179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80104e-cc66-4a1e-8e19-5fa0d4f40163 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:45.513459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50dbea1-e757-4a15-89c2-85cc876c46c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:45.514244 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50dbea1-e757-4a15-89c2-85cc876c46c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:45.514832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50dbea1-e757-4a15-89c2-85cc876c46c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:45.515252 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50dbea1-e757-4a15-89c2-85cc876c46c4 admin admin] PATH_INFO: `/v3/projects/ba69df6cb9cd4770b55fd781055b091d/users/7b3903e6fdd8465089f98cd4fcbf64d6/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:45.533670 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50dbea1-e757-4a15-89c2-85cc876c46c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba69df6cb9cd4770b55fd781055b091d, user_id=7b3903e6fdd8465089f98cd4fcbf64d6, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:45.536629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50dbea1-e757-4a15-89c2-85cc876c46c4 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:45.556568 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1007/2006] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:45 2022] PUT /identity/v3/projects/ba69df6cb9cd4770b55fd781055b091d/users/7b3903e6fdd8465089f98cd4fcbf64d6/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:21:45.560378 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9264cc3-87e4-4eb6-a08c-adb4c21d9e31 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:45.572358 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9264cc3-87e4-4eb6-a08c-adb4c21d9e31 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:45.641456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf28cb-aa53-46eb-8e7d-96502d39b4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:45.643591 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf28cb-aa53-46eb-8e7d-96502d39b4a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:45.643591 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf28cb-aa53-46eb-8e7d-96502d39b4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:45.643591 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf28cb-aa53-46eb-8e7d-96502d39b4a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:45.644134 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daf28cb-aa53-46eb-8e7d-96502d39b4a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:45.645821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daf28cb-aa53-46eb-8e7d-96502d39b4a5 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:45.673574 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1000/2007] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:45.674971 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50dbea1-e757-4a15-89c2-85cc876c46c4 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:45.690863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50dbea1-e757-4a15-89c2-85cc876c46c4 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:45.699023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97bc4b-fa63-4638-9812-7f626d17aad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:45.700228 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97bc4b-fa63-4638-9812-7f626d17aad9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:45.700590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97bc4b-fa63-4638-9812-7f626d17aad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:45.701052 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97bc4b-fa63-4638-9812-7f626d17aad9 admin admin] PATH_INFO: `/v3/projects/ba69df6cb9cd4770b55fd781055b091d/users/7b3903e6fdd8465089f98cd4fcbf64d6/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:45.703509 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee97bc4b-fa63-4638-9812-7f626d17aad9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba69df6cb9cd4770b55fd781055b091d, user_id=7b3903e6fdd8465089f98cd4fcbf64d6, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:45.719932 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee97bc4b-fa63-4638-9812-7f626d17aad9 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:45.737521 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1008/2008] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:45 2022] PUT /identity/v3/projects/ba69df6cb9cd4770b55fd781055b091d/users/7b3903e6fdd8465089f98cd4fcbf64d6/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:45.748640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed4017-ad0f-421f-b630-014a46ba03cd None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:45.748925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed4017-ad0f-421f-b630-014a46ba03cd None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:45.749198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed4017-ad0f-421f-b630-014a46ba03cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:45.787367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:45.792664 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-94ed4017-ad0f-421f-b630-014a46ba03cd None None] MFA Rules not processed for user `7b3903e6fdd8465089f98cd4fcbf64d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:45.796484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-94ed4017-ad0f-421f-b630-014a46ba03cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:45.880182 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1001/2009] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:45 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:45.891681 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97bc4b-fa63-4638-9812-7f626d17aad9 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:45.893369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-ee97bc4b-fa63-4638-9812-7f626d17aad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:45.955457 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97bc4b-fa63-4638-9812-7f626d17aad9 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:45.959123 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9969bf-05f5-40e6-9730-eb022b1f3334 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:45.959736 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9969bf-05f5-40e6-9730-eb022b1f3334 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:45.960015 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9969bf-05f5-40e6-9730-eb022b1f3334 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:45.960286 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9969bf-05f5-40e6-9730-eb022b1f3334 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:45.970222 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9969bf-05f5-40e6-9730-eb022b1f3334 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:45.971964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9969bf-05f5-40e6-9730-eb022b1f3334 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:45.982862 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1009/2010] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:45 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:21:46.060005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed4017-ad0f-421f-b630-014a46ba03cd None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:46.075617 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed4017-ad0f-421f-b630-014a46ba03cd None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:46.079378 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409db66c-624c-40e2-bf5d-151eed5ed11d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:46.079994 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409db66c-624c-40e2-bf5d-151eed5ed11d service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:46.080271 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409db66c-624c-40e2-bf5d-151eed5ed11d service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:46.080541 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409db66c-624c-40e2-bf5d-151eed5ed11d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:46.146247 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409db66c-624c-40e2-bf5d-151eed5ed11d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:46.147930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409db66c-624c-40e2-bf5d-151eed5ed11d service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:46.158666 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1002/2011] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:46 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:21:55.978552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=58,keystone:INSERT=5 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:56.033427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa2058-7201-40b5-af95-d455e24e2f9b None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:56.033704 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa2058-7201-40b5-af95-d455e24e2f9b None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:56.033982 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa2058-7201-40b5-af95-d455e24e2f9b None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:56.035406 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1010/2012] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 31 00:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:21:56.051934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbe59a-bb70-4a93-a4ad-3b61bfd8e51b None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:56.052492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbe59a-bb70-4a93-a4ad-3b61bfd8e51b None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:56.052492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbe59a-bb70-4a93-a4ad-3b61bfd8e51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:56.073135 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-2cfbe59a-bb70-4a93-a4ad-3b61bfd8e51b None None] MFA Rules not processed for user `fdacdcb2451742938164fdd44cdbaebe`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:56.075252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-2cfbe59a-bb70-4a93-a4ad-3b61bfd8e51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:56.166593 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1003/2013] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 31 00:21:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:21:56.179164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa2058-7201-40b5-af95-d455e24e2f9b None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:56.182647 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-66aa2058-7201-40b5-af95-d455e24e2f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:56.201342 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa2058-7201-40b5-af95-d455e24e2f9b None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:56.209822 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b36e7b-9ecb-4253-8858-b63af380503a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdacdcb2451742938164fdd44cdbaebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:56.210238 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b36e7b-9ecb-4253-8858-b63af380503a service placement] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:56.210521 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b36e7b-9ecb-4253-8858-b63af380503a service placement] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:56.210793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b36e7b-9ecb-4253-8858-b63af380503a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:56.221381 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b36e7b-9ecb-4253-8858-b63af380503a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:56.223230 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b36e7b-9ecb-4253-8858-b63af380503a service placement] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:56.234974 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1011/2014] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:21:56.730694 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfbe59a-bb70-4a93-a4ad-3b61bfd8e51b None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:56.732473 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-2cfbe59a-bb70-4a93-a4ad-3b61bfd8e51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:56.776937 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:56.809531 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfbe59a-bb70-4a93-a4ad-3b61bfd8e51b None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:56.819767 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aca5ce-5fd7-414a-91be-bf74a057dd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:56.822547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aca5ce-5fd7-414a-91be-bf74a057dd19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:56.822901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aca5ce-5fd7-414a-91be-bf74a057dd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:56.823290 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aca5ce-5fd7-414a-91be-bf74a057dd19 admin admin] PATH_INFO: `/v3/users/3c58b3edba70494aac0695c66f4eacd9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:56.824705 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aca5ce-5fd7-414a-91be-bf74a057dd19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c58b3edba70494aac0695c66f4eacd9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:56.826688 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aca5ce-5fd7-414a-91be-bf74a057dd19 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:56.901303 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f6aca5ce-5fd7-414a-91be-bf74a057dd19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c58b3edba70494aac0695c66f4eacd9'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:56.924706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f6aca5ce-5fd7-414a-91be-bf74a057dd19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c58b3edba70494aac0695c66f4eacd9'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:56.936373 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f6aca5ce-5fd7-414a-91be-bf74a057dd19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c58b3edba70494aac0695c66f4eacd9'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:56.944240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f6aca5ce-5fd7-414a-91be-bf74a057dd19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c58b3edba70494aac0695c66f4eacd9'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:56.944846 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f6aca5ce-5fd7-414a-91be-bf74a057dd19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c58b3edba70494aac0695c66f4eacd9'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:56.952227 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1004/2015] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:21:56 2022] DELETE /identity/v3/users/3c58b3edba70494aac0695c66f4eacd9 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:57.138825 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b36e7b-9ecb-4253-8858-b63af380503a service placement] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:57.166178 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b36e7b-9ecb-4253-8858-b63af380503a service placement] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:57.230563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f17705-39a3-4575-a9df-0743f8fda15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:57.231244 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f17705-39a3-4575-a9df-0743f8fda15c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:57.231523 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f17705-39a3-4575-a9df-0743f8fda15c admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:57.231791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f17705-39a3-4575-a9df-0743f8fda15c admin admin] PATH_INFO: `/v3/projects/ade91b4ec7ae48b78f4616fbdbf253a9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:57.232935 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f17705-39a3-4575-a9df-0743f8fda15c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ade91b4ec7ae48b78f4616fbdbf253a9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:57.234853 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f17705-39a3-4575-a9df-0743f8fda15c admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:57.308014 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-63f17705-39a3-4575-a9df-0743f8fda15c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ade91b4ec7ae48b78f4616fbdbf253a9', 'request_id': 'req-63f17705-39a3-4575-a9df-0743f8fda15c'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:57.318464 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-63f17705-39a3-4575-a9df-0743f8fda15c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ade91b4ec7ae48b78f4616fbdbf253a9', 'request_id': 'req-63f17705-39a3-4575-a9df-0743f8fda15c'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:57.329939 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-63f17705-39a3-4575-a9df-0743f8fda15c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ade91b4ec7ae48b78f4616fbdbf253a9', 'request_id': 'req-63f17705-39a3-4575-a9df-0743f8fda15c'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:57.346554 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-63f17705-39a3-4575-a9df-0743f8fda15c admin admin] The token cache is being invalidate because project ade91b4ec7ae48b78f4616fbdbf253a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:21:57.346870 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-63f17705-39a3-4575-a9df-0743f8fda15c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:57.347672 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-63f17705-39a3-4575-a9df-0743f8fda15c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:57.349442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1012/2016] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:21:57 2022] DELETE /identity/v3/projects/ade91b4ec7ae48b78f4616fbdbf253a9 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:57.354133 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aca5ce-5fd7-414a-91be-bf74a057dd19 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:57.355430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f6aca5ce-5fd7-414a-91be-bf74a057dd19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:57.371308 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aca5ce-5fd7-414a-91be-bf74a057dd19 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:57.459206 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688fcae-51f1-4e00-b024-f8ed3ba24eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:57.460270 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688fcae-51f1-4e00-b024-f8ed3ba24eaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:57.460730 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688fcae-51f1-4e00-b024-f8ed3ba24eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:57.461339 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688fcae-51f1-4e00-b024-f8ed3ba24eaa admin admin] PATH_INFO: `/v3/users/ecd42f6e02954a3f917cd8a1fe00f7aa` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:57.463697 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9688fcae-51f1-4e00-b024-f8ed3ba24eaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecd42f6e02954a3f917cd8a1fe00f7aa)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:57.467234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9688fcae-51f1-4e00-b024-f8ed3ba24eaa admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:57.548655 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9688fcae-51f1-4e00-b024-f8ed3ba24eaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecd42f6e02954a3f917cd8a1fe00f7aa'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:57.556765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:57.570016 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9688fcae-51f1-4e00-b024-f8ed3ba24eaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecd42f6e02954a3f917cd8a1fe00f7aa'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:57.579726 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9688fcae-51f1-4e00-b024-f8ed3ba24eaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecd42f6e02954a3f917cd8a1fe00f7aa'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:57.590086 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9688fcae-51f1-4e00-b024-f8ed3ba24eaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecd42f6e02954a3f917cd8a1fe00f7aa'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:57.590674 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9688fcae-51f1-4e00-b024-f8ed3ba24eaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecd42f6e02954a3f917cd8a1fe00f7aa'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:57.597292 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1005/2017] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:21:57 2022] DELETE /identity/v3/users/ecd42f6e02954a3f917cd8a1fe00f7aa => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:57.846203 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f17705-39a3-4575-a9df-0743f8fda15c admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:57.848227 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-63f17705-39a3-4575-a9df-0743f8fda15c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:57.910003 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f17705-39a3-4575-a9df-0743f8fda15c admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:58.108481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d00b1-5d2c-4ba3-8496-47f079325ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:58.109128 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d00b1-5d2c-4ba3-8496-47f079325ea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:58.109417 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d00b1-5d2c-4ba3-8496-47f079325ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:58.109692 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d00b1-5d2c-4ba3-8496-47f079325ea0 admin admin] PATH_INFO: `/v3/projects/a990dc1179c141df8082f961a8de8d42` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:58.111638 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d00b1-5d2c-4ba3-8496-47f079325ea0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a990dc1179c141df8082f961a8de8d42)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:58.113297 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d00b1-5d2c-4ba3-8496-47f079325ea0 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:58.146116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:58.271207 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-989d00b1-5d2c-4ba3-8496-47f079325ea0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a990dc1179c141df8082f961a8de8d42', 'request_id': 'req-989d00b1-5d2c-4ba3-8496-47f079325ea0'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:58.284797 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-989d00b1-5d2c-4ba3-8496-47f079325ea0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a990dc1179c141df8082f961a8de8d42', 'request_id': 'req-989d00b1-5d2c-4ba3-8496-47f079325ea0'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:58.301424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-989d00b1-5d2c-4ba3-8496-47f079325ea0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a990dc1179c141df8082f961a8de8d42', 'request_id': 'req-989d00b1-5d2c-4ba3-8496-47f079325ea0'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:58.342590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-989d00b1-5d2c-4ba3-8496-47f079325ea0 admin admin] The token cache is being invalidate because project a990dc1179c141df8082f961a8de8d42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:21:58.343176 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-989d00b1-5d2c-4ba3-8496-47f079325ea0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:58.344286 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-989d00b1-5d2c-4ba3-8496-47f079325ea0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:21:58.348141 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1013/2018] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:21:57 2022] DELETE /identity/v3/projects/a990dc1179c141df8082f961a8de8d42 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:58.355360 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2c801-200f-4ad0-bd25-171966f44a7f None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:58.356135 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2c801-200f-4ad0-bd25-171966f44a7f None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:58.356135 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2c801-200f-4ad0-bd25-171966f44a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:58.422683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-28a2c801-200f-4ad0-bd25-171966f44a7f None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:58.424251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-28a2c801-200f-4ad0-bd25-171966f44a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:58.617940 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1006/2019] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:21:58.625646 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f10519-5469-453d-bd09-951f3a6f8ac5 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:58.625931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f10519-5469-453d-bd09-951f3a6f8ac5 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:58.626270 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f10519-5469-453d-bd09-951f3a6f8ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:58.647287 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-f8f10519-5469-453d-bd09-951f3a6f8ac5 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:58.648500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f8f10519-5469-453d-bd09-951f3a6f8ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:58.657128 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1014/2020] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:58.662047 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2c801-200f-4ad0-bd25-171966f44a7f None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:58.672109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2c801-200f-4ad0-bd25-171966f44a7f None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:58.677505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f197a-bd4c-4b12-a7bd-23a62210509b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:58.678393 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f197a-bd4c-4b12-a7bd-23a62210509b admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:58.678823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f197a-bd4c-4b12-a7bd-23a62210509b admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:58.679073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f197a-bd4c-4b12-a7bd-23a62210509b admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:58.679698 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f197a-bd4c-4b12-a7bd-23a62210509b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:21:58.680538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f197a-bd4c-4b12-a7bd-23a62210509b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:58.682156 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f197a-bd4c-4b12-a7bd-23a62210509b admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:58.694796 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1007/2021] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:21:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:21:58.700371 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f10519-5469-453d-bd09-951f3a6f8ac5 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:58.709973 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f10519-5469-453d-bd09-951f3a6f8ac5 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:58.716130 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a2877-128c-489c-8b68-31ca56e4c842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:58.716795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a2877-128c-489c-8b68-31ca56e4c842 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:58.717075 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a2877-128c-489c-8b68-31ca56e4c842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:58.717350 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a2877-128c-489c-8b68-31ca56e4c842 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:58.717997 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639a2877-128c-489c-8b68-31ca56e4c842 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:58.720007 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639a2877-128c-489c-8b68-31ca56e4c842 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:58.752876 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1015/2022] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:21:58 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:58.757233 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f197a-bd4c-4b12-a7bd-23a62210509b admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:58.768549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f197a-bd4c-4b12-a7bd-23a62210509b admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:58.830684 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f6ff7-9314-41b4-a05e-e3ef0906166b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:58.831341 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f6ff7-9314-41b4-a05e-e3ef0906166b admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:58.831624 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f6ff7-9314-41b4-a05e-e3ef0906166b admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:58.831899 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f6ff7-9314-41b4-a05e-e3ef0906166b admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:58.832538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5f6ff7-9314-41b4-a05e-e3ef0906166b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:58.834469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5f6ff7-9314-41b4-a05e-e3ef0906166b admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:58.858112 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1008/2023] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:21:58 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:58.862080 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a2877-128c-489c-8b68-31ca56e4c842 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:58.873372 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a2877-128c-489c-8b68-31ca56e4c842 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:58.878014 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e0dc5-b0e7-4802-a65e-1bf0f79d0fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:58.878643 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e0dc5-b0e7-4802-a65e-1bf0f79d0fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:58.878931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e0dc5-b0e7-4802-a65e-1bf0f79d0fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:58.879213 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e0dc5-b0e7-4802-a65e-1bf0f79d0fa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:58.879855 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e0dc5-b0e7-4802-a65e-1bf0f79d0fa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:58.881450 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e0dc5-b0e7-4802-a65e-1bf0f79d0fa3 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:58.896178 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1016/2024] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:58.897846 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f6ff7-9314-41b4-a05e-e3ef0906166b admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:58.908518 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f6ff7-9314-41b4-a05e-e3ef0906166b admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:58.913306 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8205d80-d839-41f3-b542-9f3cca3f7293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:58.913904 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8205d80-d839-41f3-b542-9f3cca3f7293 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:58.914776 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8205d80-d839-41f3-b542-9f3cca3f7293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:58.915220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8205d80-d839-41f3-b542-9f3cca3f7293 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:58.915870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8205d80-d839-41f3-b542-9f3cca3f7293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:58.917463 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8205d80-d839-41f3-b542-9f3cca3f7293 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:58.928251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1009/2025] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:58.933220 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e0dc5-b0e7-4802-a65e-1bf0f79d0fa3 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:58.943466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e0dc5-b0e7-4802-a65e-1bf0f79d0fa3 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:58.948485 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd50b46-63d8-492b-8367-c36a86d68516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:58.949159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd50b46-63d8-492b-8367-c36a86d68516 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:58.949443 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd50b46-63d8-492b-8367-c36a86d68516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:58.949719 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd50b46-63d8-492b-8367-c36a86d68516 admin admin] PATH_INFO: `/v3/projects/8af1b50c2b5d4a81855ee33c3440f139/users/c15c54434b0446939c19ae25e91dba19/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:58.978421 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd50b46-63d8-492b-8367-c36a86d68516 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8af1b50c2b5d4a81855ee33c3440f139, user_id=c15c54434b0446939c19ae25e91dba19, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:58.981452 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd50b46-63d8-492b-8367-c36a86d68516 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:58.997631 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1017/2026] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:58 2022] PUT /identity/v3/projects/8af1b50c2b5d4a81855ee33c3440f139/users/c15c54434b0446939c19ae25e91dba19/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:21:59.009211 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb65ab-635f-4e00-a74d-097079ed3c5c None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:59.009497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb65ab-635f-4e00-a74d-097079ed3c5c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:59.009771 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb65ab-635f-4e00-a74d-097079ed3c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:59.040447 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-22fb65ab-635f-4e00-a74d-097079ed3c5c None None] MFA Rules not processed for user `c15c54434b0446939c19ae25e91dba19`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:59.041728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-22fb65ab-635f-4e00-a74d-097079ed3c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:59.071070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:59.119569 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1010/2027] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:59.124288 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd50b46-63d8-492b-8367-c36a86d68516 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:59.191819 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd50b46-63d8-492b-8367-c36a86d68516 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:59.198588 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154ad43-12b8-435a-a623-d7cc1e4ce958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:59.199343 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154ad43-12b8-435a-a623-d7cc1e4ce958 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:59.199702 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154ad43-12b8-435a-a623-d7cc1e4ce958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:59.200050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154ad43-12b8-435a-a623-d7cc1e4ce958 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:59.200771 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154ad43-12b8-435a-a623-d7cc1e4ce958 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:59.203200 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154ad43-12b8-435a-a623-d7cc1e4ce958 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:59.233905 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1018/2028] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:21:59 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:59.237910 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb65ab-635f-4e00-a74d-097079ed3c5c None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:59.249729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb65ab-635f-4e00-a74d-097079ed3c5c None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:59.319199 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c33360-f2b2-4e0b-bac7-032ea610eda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:59.319900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c33360-f2b2-4e0b-bac7-032ea610eda6 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:59.320243 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c33360-f2b2-4e0b-bac7-032ea610eda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:59.320619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c33360-f2b2-4e0b-bac7-032ea610eda6 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:59.321721 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c33360-f2b2-4e0b-bac7-032ea610eda6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:59.323897 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c33360-f2b2-4e0b-bac7-032ea610eda6 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:59.351296 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1011/2029] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:21:59 2022] POST /identity/v3/users => generated 294 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:21:59.355627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154ad43-12b8-435a-a623-d7cc1e4ce958 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:59.367259 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154ad43-12b8-435a-a623-d7cc1e4ce958 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:59.372554 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f16600-b7d7-45fe-802e-ac16cdaee05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:59.373173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f16600-b7d7-45fe-802e-ac16cdaee05e admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:59.373462 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f16600-b7d7-45fe-802e-ac16cdaee05e admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:59.373733 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f16600-b7d7-45fe-802e-ac16cdaee05e admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:59.374436 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f16600-b7d7-45fe-802e-ac16cdaee05e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:59.376030 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f16600-b7d7-45fe-802e-ac16cdaee05e admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:59.389219 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1019/2030] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:21:59.394075 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c33360-f2b2-4e0b-bac7-032ea610eda6 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:59.407258 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c33360-f2b2-4e0b-bac7-032ea610eda6 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:59.412488 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382b4da-b361-4e17-8f3d-b3237b678d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:59.413153 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382b4da-b361-4e17-8f3d-b3237b678d34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:59.413431 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382b4da-b361-4e17-8f3d-b3237b678d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:59.413705 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382b4da-b361-4e17-8f3d-b3237b678d34 admin admin] PATH_INFO: `/v3/projects/5440769aac0e4b828e6ca3a1c063d74b/users/3686c6674b134e68925fe38a59d6b578/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:59.444097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9382b4da-b361-4e17-8f3d-b3237b678d34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5440769aac0e4b828e6ca3a1c063d74b, user_id=3686c6674b134e68925fe38a59d6b578, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:59.447717 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9382b4da-b361-4e17-8f3d-b3237b678d34 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:59.474969 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1012/2031] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:59 2022] PUT /identity/v3/projects/5440769aac0e4b828e6ca3a1c063d74b/users/3686c6674b134e68925fe38a59d6b578/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:21:59.480065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f16600-b7d7-45fe-802e-ac16cdaee05e admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:59.492476 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f16600-b7d7-45fe-802e-ac16cdaee05e admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:59.532513 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:21:59.649992 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e49a4-3bc3-4268-8035-0ce344028ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:59.650622 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e49a4-3bc3-4268-8035-0ce344028ede admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:59.650909 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e49a4-3bc3-4268-8035-0ce344028ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:59.651250 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e49a4-3bc3-4268-8035-0ce344028ede admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:59.652083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810e49a4-3bc3-4268-8035-0ce344028ede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:59.654252 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810e49a4-3bc3-4268-8035-0ce344028ede admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:59.666090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1020/2032] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:21:59.670256 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382b4da-b361-4e17-8f3d-b3237b678d34 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:59.681851 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382b4da-b361-4e17-8f3d-b3237b678d34 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:21:59.689337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb50f7b-202e-4dae-aa41-25b2e6c0b9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:21:59.690126 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb50f7b-202e-4dae-aa41-25b2e6c0b9f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:59.690507 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb50f7b-202e-4dae-aa41-25b2e6c0b9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:59.690889 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb50f7b-202e-4dae-aa41-25b2e6c0b9f0 admin admin] PATH_INFO: `/v3/projects/5440769aac0e4b828e6ca3a1c063d74b/users/3686c6674b134e68925fe38a59d6b578/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:59.693298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb50f7b-202e-4dae-aa41-25b2e6c0b9f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5440769aac0e4b828e6ca3a1c063d74b, user_id=3686c6674b134e68925fe38a59d6b578, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:21:59.697847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb50f7b-202e-4dae-aa41-25b2e6c0b9f0 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:21:59.712328 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1013/2033] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:21:59 2022] PUT /identity/v3/projects/5440769aac0e4b828e6ca3a1c063d74b/users/3686c6674b134e68925fe38a59d6b578/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:21:59.724438 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4a607-f4c4-4b78-8ef7-c18d3a0ef13a None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:21:59.724603 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4a607-f4c4-4b78-8ef7-c18d3a0ef13a None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:21:59.725099 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4a607-f4c4-4b78-8ef7-c18d3a0ef13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:21:59.787540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-e1e4a607-f4c4-4b78-8ef7-c18d3a0ef13a None None] MFA Rules not processed for user `3686c6674b134e68925fe38a59d6b578`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:21:59.789257 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e1e4a607-f4c4-4b78-8ef7-c18d3a0ef13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:21:59.894572 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1021/2034] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:21:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:21:59.905554 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb50f7b-202e-4dae-aa41-25b2e6c0b9f0 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:21:59.907414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-ceb50f7b-202e-4dae-aa41-25b2e6c0b9f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:00.006591 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb50f7b-202e-4dae-aa41-25b2e6c0b9f0 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:00.011664 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b081fb4c-22f9-4c8a-a279-26d24d857c9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:00.012558 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081fb4c-22f9-4c8a-a279-26d24d857c9c service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:00.015364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081fb4c-22f9-4c8a-a279-26d24d857c9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:00.015663 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081fb4c-22f9-4c8a-a279-26d24d857c9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:00.069218 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:00.098873 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b081fb4c-22f9-4c8a-a279-26d24d857c9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:00.099898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b081fb4c-22f9-4c8a-a279-26d24d857c9c service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:00.110004 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1014/2035] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:21:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:22:08.393597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e4a607-f4c4-4b78-8ef7-c18d3a0ef13a None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:08.495983 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e4a607-f4c4-4b78-8ef7-c18d3a0ef13a None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:08.501710 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb21e27-209b-41ba-bcce-b69b30ce130d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:08.502367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb21e27-209b-41ba-bcce-b69b30ce130d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:08.502651 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb21e27-209b-41ba-bcce-b69b30ce130d admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:08.502934 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb21e27-209b-41ba-bcce-b69b30ce130d admin admin] PATH_INFO: `/v3/users/c15c54434b0446939c19ae25e91dba19` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:08.504174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb21e27-209b-41ba-bcce-b69b30ce130d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c15c54434b0446939c19ae25e91dba19)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:08.506168 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb21e27-209b-41ba-bcce-b69b30ce130d admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:08.552433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:08.559028 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-feb21e27-209b-41ba-bcce-b69b30ce130d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c15c54434b0446939c19ae25e91dba19'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:08.568425 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-feb21e27-209b-41ba-bcce-b69b30ce130d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c15c54434b0446939c19ae25e91dba19'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:08.577344 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-feb21e27-209b-41ba-bcce-b69b30ce130d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c15c54434b0446939c19ae25e91dba19'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:08.578139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-feb21e27-209b-41ba-bcce-b69b30ce130d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c15c54434b0446939c19ae25e91dba19'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:08.578745 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-feb21e27-209b-41ba-bcce-b69b30ce130d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c15c54434b0446939c19ae25e91dba19'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:08.597209 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1022/2036] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:22:08 2022] DELETE /identity/v3/users/c15c54434b0446939c19ae25e91dba19 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:08.602624 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c9e9d-fed8-4a1b-9054-77604f8e38df None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:08.603149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c9e9d-fed8-4a1b-9054-77604f8e38df None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:08.603270 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c9e9d-fed8-4a1b-9054-77604f8e38df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:08.626599 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-7a1c9e9d-fed8-4a1b-9054-77604f8e38df None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:08.627917 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-7a1c9e9d-fed8-4a1b-9054-77604f8e38df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:08.688802 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1015/2037] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:08.699183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb21e27-209b-41ba-bcce-b69b30ce130d admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:08.700582 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-feb21e27-209b-41ba-bcce-b69b30ce130d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:08.759161 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb21e27-209b-41ba-bcce-b69b30ce130d admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:08.763496 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954870ad-26a9-47ce-90ac-050e6664d66a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:08.764272 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954870ad-26a9-47ce-90ac-050e6664d66a service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:08.764555 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954870ad-26a9-47ce-90ac-050e6664d66a service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:08.764828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954870ad-26a9-47ce-90ac-050e6664d66a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:08.775119 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954870ad-26a9-47ce-90ac-050e6664d66a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:08.776660 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954870ad-26a9-47ce-90ac-050e6664d66a service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:08.786648 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1023/2038] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:22:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:22:09.068200 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c9e9d-fed8-4a1b-9054-77604f8e38df None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:09.069758 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-7a1c9e9d-fed8-4a1b-9054-77604f8e38df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:09.085895 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c9e9d-fed8-4a1b-9054-77604f8e38df None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:09.102570 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca401529-64dd-4ecf-a9ba-870d92d760c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:09.105813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca401529-64dd-4ecf-a9ba-870d92d760c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:09.106121 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca401529-64dd-4ecf-a9ba-870d92d760c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:09.106467 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca401529-64dd-4ecf-a9ba-870d92d760c7 admin admin] PATH_INFO: `/v3/projects/8af1b50c2b5d4a81855ee33c3440f139` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:09.107957 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca401529-64dd-4ecf-a9ba-870d92d760c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8af1b50c2b5d4a81855ee33c3440f139)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:09.110419 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca401529-64dd-4ecf-a9ba-870d92d760c7 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:09.185685 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ca401529-64dd-4ecf-a9ba-870d92d760c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8af1b50c2b5d4a81855ee33c3440f139', 'request_id': 'req-ca401529-64dd-4ecf-a9ba-870d92d760c7'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:09.193372 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ca401529-64dd-4ecf-a9ba-870d92d760c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8af1b50c2b5d4a81855ee33c3440f139', 'request_id': 'req-ca401529-64dd-4ecf-a9ba-870d92d760c7'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:09.200052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ca401529-64dd-4ecf-a9ba-870d92d760c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8af1b50c2b5d4a81855ee33c3440f139', 'request_id': 'req-ca401529-64dd-4ecf-a9ba-870d92d760c7'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:09.211443 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ca401529-64dd-4ecf-a9ba-870d92d760c7 admin admin] The token cache is being invalidate because project 8af1b50c2b5d4a81855ee33c3440f139 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:22:09.211739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ca401529-64dd-4ecf-a9ba-870d92d760c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:09.212427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ca401529-64dd-4ecf-a9ba-870d92d760c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:09.213938 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1016/2039] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:22:09 2022] DELETE /identity/v3/projects/8af1b50c2b5d4a81855ee33c3440f139 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:09.218112 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954870ad-26a9-47ce-90ac-050e6664d66a service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:09.228781 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954870ad-26a9-47ce-90ac-050e6664d66a service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:09.286371 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e827d30-1beb-420e-93fc-5f948e24e864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:09.286961 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e827d30-1beb-420e-93fc-5f948e24e864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:09.287273 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e827d30-1beb-420e-93fc-5f948e24e864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:09.287573 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e827d30-1beb-420e-93fc-5f948e24e864 admin admin] PATH_INFO: `/v3/users/3686c6674b134e68925fe38a59d6b578` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:09.288720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e827d30-1beb-420e-93fc-5f948e24e864 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3686c6674b134e68925fe38a59d6b578)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:09.291234 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e827d30-1beb-420e-93fc-5f948e24e864 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:09.337117 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-4e827d30-1beb-420e-93fc-5f948e24e864 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3686c6674b134e68925fe38a59d6b578'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:09.344183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-4e827d30-1beb-420e-93fc-5f948e24e864 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3686c6674b134e68925fe38a59d6b578'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:09.351644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-4e827d30-1beb-420e-93fc-5f948e24e864 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3686c6674b134e68925fe38a59d6b578'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:09.353165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-4e827d30-1beb-420e-93fc-5f948e24e864 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3686c6674b134e68925fe38a59d6b578'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:09.353902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-4e827d30-1beb-420e-93fc-5f948e24e864 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3686c6674b134e68925fe38a59d6b578'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:09.355074 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:09.373236 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1024/2040] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:22:09 2022] DELETE /identity/v3/users/3686c6674b134e68925fe38a59d6b578 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:09.734822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca401529-64dd-4ecf-a9ba-870d92d760c7 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:09.736389 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-ca401529-64dd-4ecf-a9ba-870d92d760c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:09.750905 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca401529-64dd-4ecf-a9ba-870d92d760c7 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:09.803413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:09.812015 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79464624-9cfe-4fba-a517-7d5ca5be79f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:09.812831 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79464624-9cfe-4fba-a517-7d5ca5be79f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:09.813267 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79464624-9cfe-4fba-a517-7d5ca5be79f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:09.813879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79464624-9cfe-4fba-a517-7d5ca5be79f8 admin admin] PATH_INFO: `/v3/projects/5440769aac0e4b828e6ca3a1c063d74b` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:09.817398 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79464624-9cfe-4fba-a517-7d5ca5be79f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5440769aac0e4b828e6ca3a1c063d74b)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:09.817398 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79464624-9cfe-4fba-a517-7d5ca5be79f8 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:09.869739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-79464624-9cfe-4fba-a517-7d5ca5be79f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5440769aac0e4b828e6ca3a1c063d74b', 'request_id': 'req-79464624-9cfe-4fba-a517-7d5ca5be79f8'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:09.877506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-79464624-9cfe-4fba-a517-7d5ca5be79f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5440769aac0e4b828e6ca3a1c063d74b', 'request_id': 'req-79464624-9cfe-4fba-a517-7d5ca5be79f8'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:09.884278 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-79464624-9cfe-4fba-a517-7d5ca5be79f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5440769aac0e4b828e6ca3a1c063d74b', 'request_id': 'req-79464624-9cfe-4fba-a517-7d5ca5be79f8'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:09.897613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-79464624-9cfe-4fba-a517-7d5ca5be79f8 admin admin] The token cache is being invalidate because project 5440769aac0e4b828e6ca3a1c063d74b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:22:09.897805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-79464624-9cfe-4fba-a517-7d5ca5be79f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:09.898460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-79464624-9cfe-4fba-a517-7d5ca5be79f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:09.900061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1017/2041] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:22:09 2022] DELETE /identity/v3/projects/5440769aac0e4b828e6ca3a1c063d74b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:09.904968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214aa30-d292-4b86-9ce5-9c446fbbfa19 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:09.905456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214aa30-d292-4b86-9ce5-9c446fbbfa19 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:09.905640 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214aa30-d292-4b86-9ce5-9c446fbbfa19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:09.925875 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-d214aa30-d292-4b86-9ce5-9c446fbbfa19 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:09.928373 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-d214aa30-d292-4b86-9ce5-9c446fbbfa19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:10.006526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1025/2042] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:10.008086 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df58cb-c7d6-47cf-9eca-e7928b887dfb None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:10.008484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df58cb-c7d6-47cf-9eca-e7928b887dfb None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:10.008768 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df58cb-c7d6-47cf-9eca-e7928b887dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:10.035692 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-f4df58cb-c7d6-47cf-9eca-e7928b887dfb None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:10.036880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f4df58cb-c7d6-47cf-9eca-e7928b887dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:10.045994 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1018/2043] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:22:10.050292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d214aa30-d292-4b86-9ce5-9c446fbbfa19 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:10.060915 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d214aa30-d292-4b86-9ce5-9c446fbbfa19 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:10.065537 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3de7aa-9ab5-4a9e-92ad-8df3a280d4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:10.066133 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3de7aa-9ab5-4a9e-92ad-8df3a280d4c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:10.066413 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3de7aa-9ab5-4a9e-92ad-8df3a280d4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:10.066683 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3de7aa-9ab5-4a9e-92ad-8df3a280d4c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:10.067365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3de7aa-9ab5-4a9e-92ad-8df3a280d4c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:22:10.067668 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3de7aa-9ab5-4a9e-92ad-8df3a280d4c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:10.069262 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3de7aa-9ab5-4a9e-92ad-8df3a280d4c5 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:10.080936 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1026/2044] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:22:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:22:10.085286 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df58cb-c7d6-47cf-9eca-e7928b887dfb None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:10.094561 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df58cb-c7d6-47cf-9eca-e7928b887dfb None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:10.099221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86563c8-aa57-4ff2-b199-f5973b664504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:10.099816 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86563c8-aa57-4ff2-b199-f5973b664504 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:10.100096 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86563c8-aa57-4ff2-b199-f5973b664504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:10.100370 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86563c8-aa57-4ff2-b199-f5973b664504 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:10.101605 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86563c8-aa57-4ff2-b199-f5973b664504 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:10.104234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86563c8-aa57-4ff2-b199-f5973b664504 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:10.131720 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1019/2045] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:22:10 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:22:10.136690 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3de7aa-9ab5-4a9e-92ad-8df3a280d4c5 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:10.147022 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3de7aa-9ab5-4a9e-92ad-8df3a280d4c5 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:10.212791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27159891-f5f4-4f11-adf2-fda7c079bda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:10.213416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27159891-f5f4-4f11-adf2-fda7c079bda7 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:10.213723 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27159891-f5f4-4f11-adf2-fda7c079bda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:10.213957 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27159891-f5f4-4f11-adf2-fda7c079bda7 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:10.214606 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27159891-f5f4-4f11-adf2-fda7c079bda7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:10.216828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27159891-f5f4-4f11-adf2-fda7c079bda7 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:10.245154 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1027/2046] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:22:10 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:22:10.246465 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86563c8-aa57-4ff2-b199-f5973b664504 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:10.257337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86563c8-aa57-4ff2-b199-f5973b664504 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:10.263299 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39459e1a-5a91-4739-bff8-df86ae491762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:10.264039 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39459e1a-5a91-4739-bff8-df86ae491762 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:10.264420 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39459e1a-5a91-4739-bff8-df86ae491762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:10.264796 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39459e1a-5a91-4739-bff8-df86ae491762 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:10.265546 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39459e1a-5a91-4739-bff8-df86ae491762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:10.267371 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39459e1a-5a91-4739-bff8-df86ae491762 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:10.280165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1020/2047] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:22:10.284367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27159891-f5f4-4f11-adf2-fda7c079bda7 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:10.295833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27159891-f5f4-4f11-adf2-fda7c079bda7 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:10.301368 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75fcea-4dc0-4d8e-8cec-d6a5886e081d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:10.301852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75fcea-4dc0-4d8e-8cec-d6a5886e081d admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:10.302229 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75fcea-4dc0-4d8e-8cec-d6a5886e081d admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:10.302498 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75fcea-4dc0-4d8e-8cec-d6a5886e081d admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:10.303175 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75fcea-4dc0-4d8e-8cec-d6a5886e081d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:10.304874 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75fcea-4dc0-4d8e-8cec-d6a5886e081d admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:10.316265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1028/2048] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:22:10.321284 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39459e1a-5a91-4739-bff8-df86ae491762 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:10.331224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39459e1a-5a91-4739-bff8-df86ae491762 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:10.335902 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f94c4-a2a4-4954-9bd5-83c00221aa19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:10.337132 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f94c4-a2a4-4954-9bd5-83c00221aa19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:10.337506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f94c4-a2a4-4954-9bd5-83c00221aa19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:10.337681 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f94c4-a2a4-4954-9bd5-83c00221aa19 admin admin] PATH_INFO: `/v3/projects/7982a3c1720e435585d5aff78cf21bf6/users/5dd83d1fec7542a1991347ee7f4f06c8/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:10.354166 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4f94c4-a2a4-4954-9bd5-83c00221aa19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7982a3c1720e435585d5aff78cf21bf6, user_id=5dd83d1fec7542a1991347ee7f4f06c8, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:10.356986 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4f94c4-a2a4-4954-9bd5-83c00221aa19 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:10.372890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1021/2049] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:22:10 2022] PUT /identity/v3/projects/7982a3c1720e435585d5aff78cf21bf6/users/5dd83d1fec7542a1991347ee7f4f06c8/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:10.382684 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937bcb8-9943-4589-a1ba-3417b085c809 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:10.383030 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937bcb8-9943-4589-a1ba-3417b085c809 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:10.384006 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937bcb8-9943-4589-a1ba-3417b085c809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:10.412269 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-9937bcb8-9943-4589-a1ba-3417b085c809 None None] MFA Rules not processed for user `5dd83d1fec7542a1991347ee7f4f06c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:10.413588 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-9937bcb8-9943-4589-a1ba-3417b085c809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:10.452305 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:10.489286 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1029/2050] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:10.501352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f94c4-a2a4-4954-9bd5-83c00221aa19 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:10.503119 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-9d4f94c4-a2a4-4954-9bd5-83c00221aa19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:10.561293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f94c4-a2a4-4954-9bd5-83c00221aa19 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:10.564951 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ce1bd-4d5a-4348-bfd8-ab1655ff2657 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:10.565561 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ce1bd-4d5a-4348-bfd8-ab1655ff2657 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:10.565843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ce1bd-4d5a-4348-bfd8-ab1655ff2657 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:10.566118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ce1bd-4d5a-4348-bfd8-ab1655ff2657 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:10.575561 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ce1bd-4d5a-4348-bfd8-ab1655ff2657 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:10.577001 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ce1bd-4d5a-4348-bfd8-ab1655ff2657 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:10.586502 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1022/2051] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:22:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:22:11.162017 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937bcb8-9943-4589-a1ba-3417b085c809 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:11.163613 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-9937bcb8-9943-4589-a1ba-3417b085c809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:11.231258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937bcb8-9943-4589-a1ba-3417b085c809 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:11.236720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1426ed1-d27a-422f-9519-6dee176fb720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:11.236720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1426ed1-d27a-422f-9519-6dee176fb720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:11.237069 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1426ed1-d27a-422f-9519-6dee176fb720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:11.237320 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1426ed1-d27a-422f-9519-6dee176fb720 admin admin] PATH_INFO: `/v3/users/4e2ab57d80f144fe8261141bd7c214b0` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:11.238467 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1426ed1-d27a-422f-9519-6dee176fb720 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e2ab57d80f144fe8261141bd7c214b0)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:11.240441 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1426ed1-d27a-422f-9519-6dee176fb720 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:11.309386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c1426ed1-d27a-422f-9519-6dee176fb720 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e2ab57d80f144fe8261141bd7c214b0'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:11.316989 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c1426ed1-d27a-422f-9519-6dee176fb720 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e2ab57d80f144fe8261141bd7c214b0'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:11.324126 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c1426ed1-d27a-422f-9519-6dee176fb720 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e2ab57d80f144fe8261141bd7c214b0'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:11.324800 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c1426ed1-d27a-422f-9519-6dee176fb720 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e2ab57d80f144fe8261141bd7c214b0'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:11.325402 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c1426ed1-d27a-422f-9519-6dee176fb720 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e2ab57d80f144fe8261141bd7c214b0'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:11.346873 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1030/2052] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:22:11 2022] DELETE /identity/v3/users/4e2ab57d80f144fe8261141bd7c214b0 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:11.351404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5b149-3d20-41f2-bc5c-d1970a04ab3a None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:11.352970 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5b149-3d20-41f2-bc5c-d1970a04ab3a None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:11.352970 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5b149-3d20-41f2-bc5c-d1970a04ab3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:11.371567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-f7d5b149-3d20-41f2-bc5c-d1970a04ab3a None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:11.372779 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f7d5b149-3d20-41f2-bc5c-d1970a04ab3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:11.404930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:11.456226 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1023/2053] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:11.466781 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1426ed1-d27a-422f-9519-6dee176fb720 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:11.468293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-c1426ed1-d27a-422f-9519-6dee176fb720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:11.537562 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1426ed1-d27a-422f-9519-6dee176fb720 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:11.542178 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d6ad1-9354-40b7-b0fb-186ada0ac7c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:11.542520 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d6ad1-9354-40b7-b0fb-186ada0ac7c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:11.542824 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d6ad1-9354-40b7-b0fb-186ada0ac7c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:11.543084 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d6ad1-9354-40b7-b0fb-186ada0ac7c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:11.553464 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d6ad1-9354-40b7-b0fb-186ada0ac7c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:11.554997 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d6ad1-9354-40b7-b0fb-186ada0ac7c6 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:11.561060 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:11.573737 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1031/2054] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:22:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:22:11.883219 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5b149-3d20-41f2-bc5c-d1970a04ab3a None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:11.885360 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f7d5b149-3d20-41f2-bc5c-d1970a04ab3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:11.909274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5b149-3d20-41f2-bc5c-d1970a04ab3a None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:11.926075 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b930c-442c-4e51-8665-3d7b26e87500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:11.928338 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b930c-442c-4e51-8665-3d7b26e87500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:11.928770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b930c-442c-4e51-8665-3d7b26e87500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:11.929149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b930c-442c-4e51-8665-3d7b26e87500 admin admin] PATH_INFO: `/v3/projects/26c75fe8d9b64910b0f238cacade3d2d` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:11.936133 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b930c-442c-4e51-8665-3d7b26e87500 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26c75fe8d9b64910b0f238cacade3d2d)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:11.938200 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b930c-442c-4e51-8665-3d7b26e87500 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:12.026906 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f76b930c-442c-4e51-8665-3d7b26e87500 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26c75fe8d9b64910b0f238cacade3d2d', 'request_id': 'req-f76b930c-442c-4e51-8665-3d7b26e87500'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:12.036561 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f76b930c-442c-4e51-8665-3d7b26e87500 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26c75fe8d9b64910b0f238cacade3d2d', 'request_id': 'req-f76b930c-442c-4e51-8665-3d7b26e87500'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:12.050384 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f76b930c-442c-4e51-8665-3d7b26e87500 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26c75fe8d9b64910b0f238cacade3d2d', 'request_id': 'req-f76b930c-442c-4e51-8665-3d7b26e87500'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:12.067615 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f76b930c-442c-4e51-8665-3d7b26e87500 admin admin] The token cache is being invalidate because project 26c75fe8d9b64910b0f238cacade3d2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:22:12.075538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f76b930c-442c-4e51-8665-3d7b26e87500 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:12.076361 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f76b930c-442c-4e51-8665-3d7b26e87500 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:12.081489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1024/2055] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:22:11 2022] DELETE /identity/v3/projects/26c75fe8d9b64910b0f238cacade3d2d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:13.947263 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d6ad1-9354-40b7-b0fb-186ada0ac7c6 service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:13.948837 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-df5d6ad1-9354-40b7-b0fb-186ada0ac7c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:13.963708 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d6ad1-9354-40b7-b0fb-186ada0ac7c6 service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:13.991296 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b930c-442c-4e51-8665-3d7b26e87500 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:13.994054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f76b930c-442c-4e51-8665-3d7b26e87500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:14.045918 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abb1a82-a807-4084-ab60-3517f66babc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:14.046752 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb1a82-a807-4084-ab60-3517f66babc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:14.047233 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb1a82-a807-4084-ab60-3517f66babc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:14.047509 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb1a82-a807-4084-ab60-3517f66babc2 admin admin] PATH_INFO: `/v3/users/5dd83d1fec7542a1991347ee7f4f06c8` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:14.048644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abb1a82-a807-4084-ab60-3517f66babc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dd83d1fec7542a1991347ee7f4f06c8)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:14.050744 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abb1a82-a807-4084-ab60-3517f66babc2 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:14.065072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b930c-442c-4e51-8665-3d7b26e87500 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:14.070532 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75e03b-8e4a-43ce-88fe-2d7bd464bc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:14.071898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75e03b-8e4a-43ce-88fe-2d7bd464bc26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:14.072223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75e03b-8e4a-43ce-88fe-2d7bd464bc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:14.072598 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75e03b-8e4a-43ce-88fe-2d7bd464bc26 admin admin] PATH_INFO: `/v3/users/04a20356b6cd44b8867b084ce0c02b32` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:14.075506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75e03b-8e4a-43ce-88fe-2d7bd464bc26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04a20356b6cd44b8867b084ce0c02b32)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:14.077459 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75e03b-8e4a-43ce-88fe-2d7bd464bc26 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:14.106682 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5abb1a82-a807-4084-ab60-3517f66babc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dd83d1fec7542a1991347ee7f4f06c8'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:14.114222 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5abb1a82-a807-4084-ab60-3517f66babc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dd83d1fec7542a1991347ee7f4f06c8'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:14.124387 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5abb1a82-a807-4084-ab60-3517f66babc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dd83d1fec7542a1991347ee7f4f06c8'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:14.125092 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5abb1a82-a807-4084-ab60-3517f66babc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dd83d1fec7542a1991347ee7f4f06c8'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:14.125734 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5abb1a82-a807-4084-ab60-3517f66babc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dd83d1fec7542a1991347ee7f4f06c8'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:14.134695 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-0e75e03b-8e4a-43ce-88fe-2d7bd464bc26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04a20356b6cd44b8867b084ce0c02b32'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:14.139629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:14.147605 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1032/2056] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:22:13 2022] DELETE /identity/v3/users/5dd83d1fec7542a1991347ee7f4f06c8 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:14.150339 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-0e75e03b-8e4a-43ce-88fe-2d7bd464bc26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04a20356b6cd44b8867b084ce0c02b32'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:14.151962 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aeca6e-1b21-4ab0-8e57-642c1ff13429 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:14.152253 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aeca6e-1b21-4ab0-8e57-642c1ff13429 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:14.152530 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aeca6e-1b21-4ab0-8e57-642c1ff13429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:14.160804 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-0e75e03b-8e4a-43ce-88fe-2d7bd464bc26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04a20356b6cd44b8867b084ce0c02b32'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:14.170785 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-0e75e03b-8e4a-43ce-88fe-2d7bd464bc26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04a20356b6cd44b8867b084ce0c02b32'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:14.171521 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-0e75e03b-8e4a-43ce-88fe-2d7bd464bc26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04a20356b6cd44b8867b084ce0c02b32'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:14.172732 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-03aeca6e-1b21-4ab0-8e57-642c1ff13429 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:14.173945 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-03aeca6e-1b21-4ab0-8e57-642c1ff13429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:14.178872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1025/2057] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:22:13 2022] DELETE /identity/v3/users/04a20356b6cd44b8867b084ce0c02b32 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:22:14.185178 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7ad37-6078-4371-aab2-040f7af4600f None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:14.185525 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7ad37-6078-4371-aab2-040f7af4600f None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:14.185811 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7ad37-6078-4371-aab2-040f7af4600f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:14.207857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-9ec7ad37-6078-4371-aab2-040f7af4600f None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:14.209058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-9ec7ad37-6078-4371-aab2-040f7af4600f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:14.235156 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1033/2058] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:14.245826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aeca6e-1b21-4ab0-8e57-642c1ff13429 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:14.247360 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-03aeca6e-1b21-4ab0-8e57-642c1ff13429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:14.268394 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:14.271509 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1026/2059] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:22:14.281908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7ad37-6078-4371-aab2-040f7af4600f None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:14.283590 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-9ec7ad37-6078-4371-aab2-040f7af4600f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:14.313783 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aeca6e-1b21-4ab0-8e57-642c1ff13429 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:14.341304 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7ad37-6078-4371-aab2-040f7af4600f None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:14.344885 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b9983-fd0b-408f-978e-08b0b1c972af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:14.345485 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b9983-fd0b-408f-978e-08b0b1c972af service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:14.345761 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b9983-fd0b-408f-978e-08b0b1c972af service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:14.346030 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b9983-fd0b-408f-978e-08b0b1c972af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:14.355353 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b9983-fd0b-408f-978e-08b0b1c972af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:14.356942 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b9983-fd0b-408f-978e-08b0b1c972af service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:14.364163 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10b484-f468-4c14-baf1-1be1e6477fb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:14.364784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10b484-f468-4c14-baf1-1be1e6477fb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:14.365200 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10b484-f468-4c14-baf1-1be1e6477fb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:14.365539 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10b484-f468-4c14-baf1-1be1e6477fb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:14.367414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1027/2060] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:22:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:22:14.376264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc10b484-f468-4c14-baf1-1be1e6477fb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:14.377744 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc10b484-f468-4c14-baf1-1be1e6477fb8 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:14.389962 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1034/2061] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:22:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:22:14.915707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b9983-fd0b-408f-978e-08b0b1c972af service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:14.917192 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-007b9983-fd0b-408f-978e-08b0b1c972af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:14.922764 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10b484-f468-4c14-baf1-1be1e6477fb8 service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:14.924180 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-fc10b484-f468-4c14-baf1-1be1e6477fb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:14.933181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b9983-fd0b-408f-978e-08b0b1c972af service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:14.938036 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ebdbca-1caf-4f24-90d1-d7f3dde49a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:14.938815 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebdbca-1caf-4f24-90d1-d7f3dde49a7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:14.939138 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebdbca-1caf-4f24-90d1-d7f3dde49a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:14.939419 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebdbca-1caf-4f24-90d1-d7f3dde49a7f admin admin] PATH_INFO: `/v3/projects/6b680d27f22e41bab009dc80a23d6496` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:14.940605 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ebdbca-1caf-4f24-90d1-d7f3dde49a7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b680d27f22e41bab009dc80a23d6496)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:14.942531 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ebdbca-1caf-4f24-90d1-d7f3dde49a7f admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:14.960009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10b484-f468-4c14-baf1-1be1e6477fb8 service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:14.966245 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f66c4c0-0781-4515-862a-cda6e66ebbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:14.967197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66c4c0-0781-4515-862a-cda6e66ebbee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:14.967488 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66c4c0-0781-4515-862a-cda6e66ebbee admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:14.967885 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66c4c0-0781-4515-862a-cda6e66ebbee admin admin] PATH_INFO: `/v3/projects/7982a3c1720e435585d5aff78cf21bf6` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:14.974730 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f66c4c0-0781-4515-862a-cda6e66ebbee admin admin] RBAC: Authorizing `identity:delete_project(project_id=7982a3c1720e435585d5aff78cf21bf6)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:14.976995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f66c4c0-0781-4515-862a-cda6e66ebbee admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:15.002365 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-c1ebdbca-1caf-4f24-90d1-d7f3dde49a7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b680d27f22e41bab009dc80a23d6496', 'request_id': 'req-c1ebdbca-1caf-4f24-90d1-d7f3dde49a7f'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:15.010315 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-c1ebdbca-1caf-4f24-90d1-d7f3dde49a7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b680d27f22e41bab009dc80a23d6496', 'request_id': 'req-c1ebdbca-1caf-4f24-90d1-d7f3dde49a7f'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:15.016549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-c1ebdbca-1caf-4f24-90d1-d7f3dde49a7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b680d27f22e41bab009dc80a23d6496', 'request_id': 'req-c1ebdbca-1caf-4f24-90d1-d7f3dde49a7f'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:15.018380 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:15.028516 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8f66c4c0-0781-4515-862a-cda6e66ebbee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7982a3c1720e435585d5aff78cf21bf6', 'request_id': 'req-8f66c4c0-0781-4515-862a-cda6e66ebbee'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:15.032925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-c1ebdbca-1caf-4f24-90d1-d7f3dde49a7f admin admin] The token cache is being invalidate because project 6b680d27f22e41bab009dc80a23d6496 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:22:15.033306 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-c1ebdbca-1caf-4f24-90d1-d7f3dde49a7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:15.034167 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-c1ebdbca-1caf-4f24-90d1-d7f3dde49a7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:15.035946 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1028/2062] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:22:14 2022] DELETE /identity/v3/projects/6b680d27f22e41bab009dc80a23d6496 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:15.036060 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8f66c4c0-0781-4515-862a-cda6e66ebbee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7982a3c1720e435585d5aff78cf21bf6', 'request_id': 'req-8f66c4c0-0781-4515-862a-cda6e66ebbee'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:15.039714 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ebdbca-1caf-4f24-90d1-d7f3dde49a7f admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:15.041058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c1ebdbca-1caf-4f24-90d1-d7f3dde49a7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:15.043502 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8f66c4c0-0781-4515-862a-cda6e66ebbee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7982a3c1720e435585d5aff78cf21bf6', 'request_id': 'req-8f66c4c0-0781-4515-862a-cda6e66ebbee'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:15.060512 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8f66c4c0-0781-4515-862a-cda6e66ebbee admin admin] The token cache is being invalidate because project 7982a3c1720e435585d5aff78cf21bf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:22:15.060895 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8f66c4c0-0781-4515-862a-cda6e66ebbee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:15.061698 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8f66c4c0-0781-4515-862a-cda6e66ebbee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:15.063318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1035/2063] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:22:14 2022] DELETE /identity/v3/projects/7982a3c1720e435585d5aff78cf21bf6 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:22:15.068099 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0871f-4a77-4400-ab3a-b5930c7eb500 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:15.068564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0871f-4a77-4400-ab3a-b5930c7eb500 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:15.068888 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0871f-4a77-4400-ab3a-b5930c7eb500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:15.090760 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-dad0871f-4a77-4400-ab3a-b5930c7eb500 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:15.092105 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-dad0871f-4a77-4400-ab3a-b5930c7eb500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:15.115121 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ebdbca-1caf-4f24-90d1-d7f3dde49a7f admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:15.159823 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:15.172138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1036/2064] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:22:15.179677 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210f19a-0f75-43e1-b08e-bdc3ce62c366 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:15.180144 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210f19a-0f75-43e1-b08e-bdc3ce62c366 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:15.181041 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210f19a-0f75-43e1-b08e-bdc3ce62c366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:15.189770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355329e0-d9fe-43bd-b81c-73cdac1360c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:15.191258 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355329e0-d9fe-43bd-b81c-73cdac1360c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:15.191547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355329e0-d9fe-43bd-b81c-73cdac1360c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:15.191830 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355329e0-d9fe-43bd-b81c-73cdac1360c9 admin admin] PATH_INFO: `/v3/users/7b3903e6fdd8465089f98cd4fcbf64d6` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:15.193054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355329e0-d9fe-43bd-b81c-73cdac1360c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b3903e6fdd8465089f98cd4fcbf64d6)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:15.195478 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355329e0-d9fe-43bd-b81c-73cdac1360c9 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:15.202085 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-0210f19a-0f75-43e1-b08e-bdc3ce62c366 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:15.203321 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-0210f19a-0f75-43e1-b08e-bdc3ce62c366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:15.211335 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1037/2065] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:22:15.215859 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210f19a-0f75-43e1-b08e-bdc3ce62c366 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:15.224733 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210f19a-0f75-43e1-b08e-bdc3ce62c366 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:15.231693 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373bab03-3f0c-48dc-b8c3-7fee01e4cd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:15.232402 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bab03-3f0c-48dc-b8c3-7fee01e4cd97 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:15.232685 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bab03-3f0c-48dc-b8c3-7fee01e4cd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:15.232958 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bab03-3f0c-48dc-b8c3-7fee01e4cd97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:15.233598 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373bab03-3f0c-48dc-b8c3-7fee01e4cd97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:22:15.233948 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373bab03-3f0c-48dc-b8c3-7fee01e4cd97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:15.236136 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373bab03-3f0c-48dc-b8c3-7fee01e4cd97 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:15.249598 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1038/2066] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:22:15.251123 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-355329e0-d9fe-43bd-b81c-73cdac1360c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b3903e6fdd8465089f98cd4fcbf64d6'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:15.253916 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373bab03-3f0c-48dc-b8c3-7fee01e4cd97 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:15.264101 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373bab03-3f0c-48dc-b8c3-7fee01e4cd97 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:15.264238 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-355329e0-d9fe-43bd-b81c-73cdac1360c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b3903e6fdd8465089f98cd4fcbf64d6'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:15.268842 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b5675-833e-4976-bfa0-8d667a884acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:15.269420 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b5675-833e-4976-bfa0-8d667a884acc admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:15.269708 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b5675-833e-4976-bfa0-8d667a884acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:15.269959 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b5675-833e-4976-bfa0-8d667a884acc admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:15.270580 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b5675-833e-4976-bfa0-8d667a884acc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:15.271449 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-355329e0-d9fe-43bd-b81c-73cdac1360c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b3903e6fdd8465089f98cd4fcbf64d6'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:15.272552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b5675-833e-4976-bfa0-8d667a884acc admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:15.279084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-355329e0-d9fe-43bd-b81c-73cdac1360c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b3903e6fdd8465089f98cd4fcbf64d6'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:15.279716 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-355329e0-d9fe-43bd-b81c-73cdac1360c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b3903e6fdd8465089f98cd4fcbf64d6'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:15.285804 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1029/2067] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:22:15 2022] DELETE /identity/v3/users/7b3903e6fdd8465089f98cd4fcbf64d6 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:22:15.305286 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1039/2068] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:22:15 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:22:15.307080 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355329e0-d9fe-43bd-b81c-73cdac1360c9 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:15.308492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-355329e0-d9fe-43bd-b81c-73cdac1360c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:15.335999 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:15.379711 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355329e0-d9fe-43bd-b81c-73cdac1360c9 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:15.453246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff06f29-3bd5-4e8b-8162-ad3d626868b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:15.453871 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff06f29-3bd5-4e8b-8162-ad3d626868b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:15.454172 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff06f29-3bd5-4e8b-8162-ad3d626868b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:15.454450 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff06f29-3bd5-4e8b-8162-ad3d626868b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:15.455130 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff06f29-3bd5-4e8b-8162-ad3d626868b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:15.457052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff06f29-3bd5-4e8b-8162-ad3d626868b4 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:15.490123 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1030/2069] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:22:15 2022] POST /identity/v3/users => generated 296 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:22:15.491567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b5675-833e-4976-bfa0-8d667a884acc admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:15.508903 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b5675-833e-4976-bfa0-8d667a884acc admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:15.514499 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db185f2-8530-4304-b041-27cb89a59c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:15.515002 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db185f2-8530-4304-b041-27cb89a59c07 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:15.515327 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db185f2-8530-4304-b041-27cb89a59c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:15.515748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db185f2-8530-4304-b041-27cb89a59c07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:15.516879 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db185f2-8530-4304-b041-27cb89a59c07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:15.518140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db185f2-8530-4304-b041-27cb89a59c07 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:15.529342 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1040/2070] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:22:15.533330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff06f29-3bd5-4e8b-8162-ad3d626868b4 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:15.545060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff06f29-3bd5-4e8b-8162-ad3d626868b4 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:15.550104 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93d087-8a40-4f8f-9656-19e32693d87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:15.550712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93d087-8a40-4f8f-9656-19e32693d87a admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:15.551643 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93d087-8a40-4f8f-9656-19e32693d87a admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:15.551643 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93d087-8a40-4f8f-9656-19e32693d87a admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:15.551937 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93d087-8a40-4f8f-9656-19e32693d87a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:15.553528 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93d087-8a40-4f8f-9656-19e32693d87a admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:15.569750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1031/2071] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:22:15.576487 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db185f2-8530-4304-b041-27cb89a59c07 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:15.588324 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db185f2-8530-4304-b041-27cb89a59c07 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:15.591960 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93d087-8a40-4f8f-9656-19e32693d87a admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:15.593771 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-1f93d087-8a40-4f8f-9656-19e32693d87a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:15.593991 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64833bb5-10cf-4b2d-8d41-1cff8100971f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:15.594675 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64833bb5-10cf-4b2d-8d41-1cff8100971f admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:15.595119 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64833bb5-10cf-4b2d-8d41-1cff8100971f admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:15.595535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64833bb5-10cf-4b2d-8d41-1cff8100971f admin admin] PATH_INFO: `/v3/projects/1b066d1051854f6689c8c6c26ce82000/users/d98b14966ff0479bafea6b1d7c038581/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:15.609057 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93d087-8a40-4f8f-9656-19e32693d87a admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:15.613334 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64833bb5-10cf-4b2d-8d41-1cff8100971f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b066d1051854f6689c8c6c26ce82000, user_id=d98b14966ff0479bafea6b1d7c038581, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:15.613828 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7787e-d90d-4de5-b85f-4b7d8afc80f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:15.614462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7787e-d90d-4de5-b85f-4b7d8afc80f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:15.614757 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7787e-d90d-4de5-b85f-4b7d8afc80f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:15.615025 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7787e-d90d-4de5-b85f-4b7d8afc80f8 admin admin] PATH_INFO: `/v3/projects/ba69df6cb9cd4770b55fd781055b091d` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:15.616220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7787e-d90d-4de5-b85f-4b7d8afc80f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba69df6cb9cd4770b55fd781055b091d)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:15.616435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64833bb5-10cf-4b2d-8d41-1cff8100971f admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:15.619140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7787e-d90d-4de5-b85f-4b7d8afc80f8 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:15.636832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1041/2072] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:22:15 2022] PUT /identity/v3/projects/1b066d1051854f6689c8c6c26ce82000/users/d98b14966ff0479bafea6b1d7c038581/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:22:15.648909 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da474a-4777-4651-8e88-7e7bf4020fcc None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:15.649744 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da474a-4777-4651-8e88-7e7bf4020fcc None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:15.650105 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da474a-4777-4651-8e88-7e7bf4020fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:15.671195 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-a2f7787e-d90d-4de5-b85f-4b7d8afc80f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba69df6cb9cd4770b55fd781055b091d', 'request_id': 'req-a2f7787e-d90d-4de5-b85f-4b7d8afc80f8'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:15.678974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-a2f7787e-d90d-4de5-b85f-4b7d8afc80f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba69df6cb9cd4770b55fd781055b091d', 'request_id': 'req-a2f7787e-d90d-4de5-b85f-4b7d8afc80f8'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:15.680448 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-a2da474a-4777-4651-8e88-7e7bf4020fcc None None] MFA Rules not processed for user `d98b14966ff0479bafea6b1d7c038581`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:15.681660 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a2da474a-4777-4651-8e88-7e7bf4020fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:15.685581 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-a2f7787e-d90d-4de5-b85f-4b7d8afc80f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba69df6cb9cd4770b55fd781055b091d', 'request_id': 'req-a2f7787e-d90d-4de5-b85f-4b7d8afc80f8'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:15.700463 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-a2f7787e-d90d-4de5-b85f-4b7d8afc80f8 admin admin] The token cache is being invalidate because project ba69df6cb9cd4770b55fd781055b091d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:22:15.700894 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-a2f7787e-d90d-4de5-b85f-4b7d8afc80f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:15.702600 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-a2f7787e-d90d-4de5-b85f-4b7d8afc80f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:15.704222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1032/2073] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:22:15 2022] DELETE /identity/v3/projects/ba69df6cb9cd4770b55fd781055b091d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:22:15.709218 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe87c5-9771-4a93-bfe6-a9367db9fcf7 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:15.709872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe87c5-9771-4a93-bfe6-a9367db9fcf7 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:15.709872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe87c5-9771-4a93-bfe6-a9367db9fcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:15.738630 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-affe87c5-9771-4a93-bfe6-a9367db9fcf7 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:15.740140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-affe87c5-9771-4a93-bfe6-a9367db9fcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:15.757830 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1042/2074] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:15 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:22:15.770950 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da474a-4777-4651-8e88-7e7bf4020fcc None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:15.772390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a2da474a-4777-4651-8e88-7e7bf4020fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:15.773215 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:15.818437 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1033/2075] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:15.826240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597db92-4c76-4691-b6fb-659257bb17dd None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:15.826528 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597db92-4c76-4691-b6fb-659257bb17dd None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:15.826802 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597db92-4c76-4691-b6fb-659257bb17dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:15.834397 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da474a-4777-4651-8e88-7e7bf4020fcc None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:15.839986 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a418415-4e77-4dd8-88a1-e8a61c0fc141 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:15.840572 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a418415-4e77-4dd8-88a1-e8a61c0fc141 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:15.840856 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a418415-4e77-4dd8-88a1-e8a61c0fc141 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:15.841130 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a418415-4e77-4dd8-88a1-e8a61c0fc141 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:15.850332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-e597db92-4c76-4691-b6fb-659257bb17dd None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:15.850918 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a418415-4e77-4dd8-88a1-e8a61c0fc141 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:15.852700 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-e597db92-4c76-4691-b6fb-659257bb17dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:15.853157 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a418415-4e77-4dd8-88a1-e8a61c0fc141 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:15.862584 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1043/2076] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:22:15 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:22:15.866575 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1034/2077] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:22:15.867840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a418415-4e77-4dd8-88a1-e8a61c0fc141 service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:15.878816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a418415-4e77-4dd8-88a1-e8a61c0fc141 service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:15.894798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0628a54-c58b-4e4a-a5ca-4cc22fecb078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:15.895465 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0628a54-c58b-4e4a-a5ca-4cc22fecb078 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:15.895748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0628a54-c58b-4e4a-a5ca-4cc22fecb078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:15.896022 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0628a54-c58b-4e4a-a5ca-4cc22fecb078 admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:15.896673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0628a54-c58b-4e4a-a5ca-4cc22fecb078 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:22:15.896982 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0628a54-c58b-4e4a-a5ca-4cc22fecb078 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:15.898724 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0628a54-c58b-4e4a-a5ca-4cc22fecb078 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:15.912613 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1044/2078] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:22:15.916314 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597db92-4c76-4691-b6fb-659257bb17dd None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:15.925472 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597db92-4c76-4691-b6fb-659257bb17dd None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:15.930227 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec3e09-e47b-4271-a562-b1bbfb7b00fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:15.930843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec3e09-e47b-4271-a562-b1bbfb7b00fa admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:15.931886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec3e09-e47b-4271-a562-b1bbfb7b00fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:15.931886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec3e09-e47b-4271-a562-b1bbfb7b00fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:15.933080 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec3e09-e47b-4271-a562-b1bbfb7b00fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:15.934981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec3e09-e47b-4271-a562-b1bbfb7b00fa admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:15.966947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1035/2079] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:22:15 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:22:15.971579 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0628a54-c58b-4e4a-a5ca-4cc22fecb078 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:15.977726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:15.983628 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0628a54-c58b-4e4a-a5ca-4cc22fecb078 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:16.046114 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41527f7-12da-4dc0-908a-b7ea27225216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:16.046728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41527f7-12da-4dc0-908a-b7ea27225216 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:16.047009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41527f7-12da-4dc0-908a-b7ea27225216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:16.047357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41527f7-12da-4dc0-908a-b7ea27225216 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:16.047945 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41527f7-12da-4dc0-908a-b7ea27225216 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:16.049869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41527f7-12da-4dc0-908a-b7ea27225216 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:16.071842 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1045/2080] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:22:15 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:22:16.077649 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec3e09-e47b-4271-a562-b1bbfb7b00fa admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:16.088945 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec3e09-e47b-4271-a562-b1bbfb7b00fa admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:16.093775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d099c75-3070-48fd-9f05-5efa254584ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:16.094322 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d099c75-3070-48fd-9f05-5efa254584ee admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:16.094600 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d099c75-3070-48fd-9f05-5efa254584ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:16.094875 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d099c75-3070-48fd-9f05-5efa254584ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:16.095549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d099c75-3070-48fd-9f05-5efa254584ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:16.097137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d099c75-3070-48fd-9f05-5efa254584ee admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:16.108528 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1036/2081] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:22:16.113028 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41527f7-12da-4dc0-908a-b7ea27225216 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:16.123662 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41527f7-12da-4dc0-908a-b7ea27225216 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:16.128480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d924314f-2c00-425a-b0c3-33d52b5e396d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:16.129082 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924314f-2c00-425a-b0c3-33d52b5e396d admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:16.129361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924314f-2c00-425a-b0c3-33d52b5e396d admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:16.129631 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924314f-2c00-425a-b0c3-33d52b5e396d admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:16.130271 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d924314f-2c00-425a-b0c3-33d52b5e396d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:16.131937 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d924314f-2c00-425a-b0c3-33d52b5e396d admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:16.145504 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1046/2082] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:22:16.155026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d099c75-3070-48fd-9f05-5efa254584ee admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:16.166056 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d099c75-3070-48fd-9f05-5efa254584ee admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:16.172744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e81e0-ff0f-45ec-9b91-62665ad20363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:16.173406 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e81e0-ff0f-45ec-9b91-62665ad20363 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:16.173688 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e81e0-ff0f-45ec-9b91-62665ad20363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:16.173962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e81e0-ff0f-45ec-9b91-62665ad20363 admin admin] PATH_INFO: `/v3/projects/4d10b95d32754ac5975f7f00723bd83d/users/03d170cb2edb4b37a09465a52b9e3a6e/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:16.193111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24e81e0-ff0f-45ec-9b91-62665ad20363 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d10b95d32754ac5975f7f00723bd83d, user_id=03d170cb2edb4b37a09465a52b9e3a6e, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:16.197050 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24e81e0-ff0f-45ec-9b91-62665ad20363 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:16.220184 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1037/2083] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:22:16 2022] PUT /identity/v3/projects/4d10b95d32754ac5975f7f00723bd83d/users/03d170cb2edb4b37a09465a52b9e3a6e/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:22:16.232451 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55caa365-0f5e-437f-9931-6a2093d351be None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:16.233075 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55caa365-0f5e-437f-9931-6a2093d351be None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:16.233426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55caa365-0f5e-437f-9931-6a2093d351be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:16.265449 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-55caa365-0f5e-437f-9931-6a2093d351be None None] MFA Rules not processed for user `03d170cb2edb4b37a09465a52b9e3a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:16.266746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-55caa365-0f5e-437f-9931-6a2093d351be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:16.341535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1047/2084] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:16.345943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e81e0-ff0f-45ec-9b91-62665ad20363 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:16.413575 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e81e0-ff0f-45ec-9b91-62665ad20363 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:16.419590 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d21f8e-3fda-4718-baab-660249e422d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:16.420199 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d21f8e-3fda-4718-baab-660249e422d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:16.420480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d21f8e-3fda-4718-baab-660249e422d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:16.420755 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d21f8e-3fda-4718-baab-660249e422d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:16.421404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d21f8e-3fda-4718-baab-660249e422d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:16.423363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d21f8e-3fda-4718-baab-660249e422d7 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:16.450083 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1038/2085] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:22:16 2022] POST /identity/v3/projects => generated 369 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:22:16.454159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55caa365-0f5e-437f-9931-6a2093d351be None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:16.465639 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55caa365-0f5e-437f-9931-6a2093d351be None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:16.522859 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775df8a5-0508-4a59-a15b-e5c939ca3dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:16.523500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775df8a5-0508-4a59-a15b-e5c939ca3dcf admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:16.523782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775df8a5-0508-4a59-a15b-e5c939ca3dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:16.524586 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775df8a5-0508-4a59-a15b-e5c939ca3dcf admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:16.525499 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775df8a5-0508-4a59-a15b-e5c939ca3dcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:16.527501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775df8a5-0508-4a59-a15b-e5c939ca3dcf admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:16.544824 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:16.550745 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1048/2086] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:22:16 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:22:16.554638 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d21f8e-3fda-4718-baab-660249e422d7 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:16.565614 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d21f8e-3fda-4718-baab-660249e422d7 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:16.571989 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8e08e-c34c-4163-aaba-16885a8d9654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:16.572594 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8e08e-c34c-4163-aaba-16885a8d9654 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:16.572876 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8e08e-c34c-4163-aaba-16885a8d9654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:16.573149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8e08e-c34c-4163-aaba-16885a8d9654 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:16.573795 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e8e08e-c34c-4163-aaba-16885a8d9654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:16.575441 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e8e08e-c34c-4163-aaba-16885a8d9654 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:16.588720 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1039/2087] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:22:16.593399 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775df8a5-0508-4a59-a15b-e5c939ca3dcf admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:16.607468 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775df8a5-0508-4a59-a15b-e5c939ca3dcf admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:16.612372 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3783e99d-ac4d-44e8-8056-76ee5fb4171a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:16.613034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783e99d-ac4d-44e8-8056-76ee5fb4171a admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:16.613316 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783e99d-ac4d-44e8-8056-76ee5fb4171a admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:16.613771 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783e99d-ac4d-44e8-8056-76ee5fb4171a admin admin] PATH_INFO: `/v3/projects/9259f222ca8f4eb5bb6a72415593f0e4/users/2efc99aac4564f538819203bbe5d8865/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:16.631067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3783e99d-ac4d-44e8-8056-76ee5fb4171a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9259f222ca8f4eb5bb6a72415593f0e4, user_id=2efc99aac4564f538819203bbe5d8865, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:16.635596 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3783e99d-ac4d-44e8-8056-76ee5fb4171a admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:16.650945 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1049/2088] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:22:16 2022] PUT /identity/v3/projects/9259f222ca8f4eb5bb6a72415593f0e4/users/2efc99aac4564f538819203bbe5d8865/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:22:16.654923 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8e08e-c34c-4163-aaba-16885a8d9654 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:16.666071 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8e08e-c34c-4163-aaba-16885a8d9654 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:16.706579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:16.724883 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885acf44-1158-434d-9e9c-0d837b67491f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:16.725484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885acf44-1158-434d-9e9c-0d837b67491f admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:16.725786 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885acf44-1158-434d-9e9c-0d837b67491f admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:16.726056 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885acf44-1158-434d-9e9c-0d837b67491f admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:16.726691 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885acf44-1158-434d-9e9c-0d837b67491f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:16.728902 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885acf44-1158-434d-9e9c-0d837b67491f admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:16.741917 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1040/2089] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:22:16.743350 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3783e99d-ac4d-44e8-8056-76ee5fb4171a admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:16.753971 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3783e99d-ac4d-44e8-8056-76ee5fb4171a admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:16.760020 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6e855-fc00-482d-8176-0eb224f8d994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:16.760681 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6e855-fc00-482d-8176-0eb224f8d994 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:16.760964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6e855-fc00-482d-8176-0eb224f8d994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:16.761236 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6e855-fc00-482d-8176-0eb224f8d994 admin admin] PATH_INFO: `/v3/projects/9259f222ca8f4eb5bb6a72415593f0e4/users/2efc99aac4564f538819203bbe5d8865/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:16.764864 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6e855-fc00-482d-8176-0eb224f8d994 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9259f222ca8f4eb5bb6a72415593f0e4, user_id=2efc99aac4564f538819203bbe5d8865, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:16.767599 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6e855-fc00-482d-8176-0eb224f8d994 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:16.779616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1050/2090] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:22:16 2022] PUT /identity/v3/projects/9259f222ca8f4eb5bb6a72415593f0e4/users/2efc99aac4564f538819203bbe5d8865/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:16.789971 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d58ca-c63d-46d8-8420-1fb28123d51e None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:16.790500 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d58ca-c63d-46d8-8420-1fb28123d51e None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:16.790934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d58ca-c63d-46d8-8420-1fb28123d51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:16.819450 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-bc6d58ca-c63d-46d8-8420-1fb28123d51e None None] MFA Rules not processed for user `2efc99aac4564f538819203bbe5d8865`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:16.820692 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-bc6d58ca-c63d-46d8-8420-1fb28123d51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:16.890415 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1041/2091] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:16 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:16.902228 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6e855-fc00-482d-8176-0eb224f8d994 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:16.904033 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-5aa6e855-fc00-482d-8176-0eb224f8d994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:16.968148 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6e855-fc00-482d-8176-0eb224f8d994 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:16.974391 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbb2c5-e192-4e02-a43f-4b29a72c43e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:16.975691 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbb2c5-e192-4e02-a43f-4b29a72c43e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:16.976280 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbb2c5-e192-4e02-a43f-4b29a72c43e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:16.976550 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbb2c5-e192-4e02-a43f-4b29a72c43e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:16.986599 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdbb2c5-e192-4e02-a43f-4b29a72c43e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:16.988290 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdbb2c5-e192-4e02-a43f-4b29a72c43e7 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:16.999526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1051/2092] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:22:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:22:18.438983 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d58ca-c63d-46d8-8420-1fb28123d51e None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:18.501772 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d58ca-c63d-46d8-8420-1fb28123d51e None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:18.506390 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71469d0b-fd9b-4fac-a0c2-5c8a303219e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:18.507023 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71469d0b-fd9b-4fac-a0c2-5c8a303219e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:18.507333 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71469d0b-fd9b-4fac-a0c2-5c8a303219e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:18.507604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71469d0b-fd9b-4fac-a0c2-5c8a303219e8 admin admin] PATH_INFO: `/v3/users/03d170cb2edb4b37a09465a52b9e3a6e` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:18.509255 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71469d0b-fd9b-4fac-a0c2-5c8a303219e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03d170cb2edb4b37a09465a52b9e3a6e)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:18.511406 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71469d0b-fd9b-4fac-a0c2-5c8a303219e8 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:18.558443 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-71469d0b-fd9b-4fac-a0c2-5c8a303219e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03d170cb2edb4b37a09465a52b9e3a6e'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:18.570557 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-71469d0b-fd9b-4fac-a0c2-5c8a303219e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03d170cb2edb4b37a09465a52b9e3a6e'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:18.572554 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:18.579725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-71469d0b-fd9b-4fac-a0c2-5c8a303219e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03d170cb2edb4b37a09465a52b9e3a6e'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:18.587428 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-71469d0b-fd9b-4fac-a0c2-5c8a303219e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03d170cb2edb4b37a09465a52b9e3a6e'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:18.588050 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-71469d0b-fd9b-4fac-a0c2-5c8a303219e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03d170cb2edb4b37a09465a52b9e3a6e'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:18.594057 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1042/2093] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:22:18 2022] DELETE /identity/v3/users/03d170cb2edb4b37a09465a52b9e3a6e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:18.598578 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc1caa-06ab-426e-8363-3b810b71bc03 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:18.599060 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc1caa-06ab-426e-8363-3b810b71bc03 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:18.599178 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc1caa-06ab-426e-8363-3b810b71bc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:18.619357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-60dc1caa-06ab-426e-8363-3b810b71bc03 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:18.620569 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-60dc1caa-06ab-426e-8363-3b810b71bc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:18.679878 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1052/2094] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:18.690350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71469d0b-fd9b-4fac-a0c2-5c8a303219e8 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:18.691770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-71469d0b-fd9b-4fac-a0c2-5c8a303219e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:18.752220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71469d0b-fd9b-4fac-a0c2-5c8a303219e8 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:18.755889 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d3a56-5f57-42f6-9e8c-e2762be2f8a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:18.756497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d3a56-5f57-42f6-9e8c-e2762be2f8a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:18.756780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d3a56-5f57-42f6-9e8c-e2762be2f8a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:18.757050 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d3a56-5f57-42f6-9e8c-e2762be2f8a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:18.767337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d3a56-5f57-42f6-9e8c-e2762be2f8a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:18.768758 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d3a56-5f57-42f6-9e8c-e2762be2f8a8 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:18.778499 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1043/2095] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:22:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:22:19.044724 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc1caa-06ab-426e-8363-3b810b71bc03 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:19.046268 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-60dc1caa-06ab-426e-8363-3b810b71bc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:19.061880 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc1caa-06ab-426e-8363-3b810b71bc03 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:19.066834 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e82f1-bd27-4acb-8a8c-44deaa804c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:19.067719 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e82f1-bd27-4acb-8a8c-44deaa804c28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:19.068015 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e82f1-bd27-4acb-8a8c-44deaa804c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:19.068289 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e82f1-bd27-4acb-8a8c-44deaa804c28 admin admin] PATH_INFO: `/v3/projects/4d10b95d32754ac5975f7f00723bd83d` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:19.069523 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e82f1-bd27-4acb-8a8c-44deaa804c28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d10b95d32754ac5975f7f00723bd83d)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:19.072049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e82f1-bd27-4acb-8a8c-44deaa804c28 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:19.117668 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aa5e82f1-bd27-4acb-8a8c-44deaa804c28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d10b95d32754ac5975f7f00723bd83d', 'request_id': 'req-aa5e82f1-bd27-4acb-8a8c-44deaa804c28'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:19.123377 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aa5e82f1-bd27-4acb-8a8c-44deaa804c28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d10b95d32754ac5975f7f00723bd83d', 'request_id': 'req-aa5e82f1-bd27-4acb-8a8c-44deaa804c28'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:19.125209 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:19.129942 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aa5e82f1-bd27-4acb-8a8c-44deaa804c28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d10b95d32754ac5975f7f00723bd83d', 'request_id': 'req-aa5e82f1-bd27-4acb-8a8c-44deaa804c28'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:19.144354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aa5e82f1-bd27-4acb-8a8c-44deaa804c28 admin admin] The token cache is being invalidate because project 4d10b95d32754ac5975f7f00723bd83d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:22:19.144645 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aa5e82f1-bd27-4acb-8a8c-44deaa804c28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:19.145284 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aa5e82f1-bd27-4acb-8a8c-44deaa804c28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:19.146799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1053/2096] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:22:19 2022] DELETE /identity/v3/projects/4d10b95d32754ac5975f7f00723bd83d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:19.151685 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d3a56-5f57-42f6-9e8c-e2762be2f8a8 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:19.161309 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d3a56-5f57-42f6-9e8c-e2762be2f8a8 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:19.218528 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c347d88-47cf-4416-b8f7-9c204bbd8f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:19.219098 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c347d88-47cf-4416-b8f7-9c204bbd8f7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:19.219378 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c347d88-47cf-4416-b8f7-9c204bbd8f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:19.219647 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c347d88-47cf-4416-b8f7-9c204bbd8f7a admin admin] PATH_INFO: `/v3/users/2efc99aac4564f538819203bbe5d8865` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:19.220755 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c347d88-47cf-4416-b8f7-9c204bbd8f7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2efc99aac4564f538819203bbe5d8865)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:19.222665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c347d88-47cf-4416-b8f7-9c204bbd8f7a admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:19.271387 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9c347d88-47cf-4416-b8f7-9c204bbd8f7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2efc99aac4564f538819203bbe5d8865'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:19.284627 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9c347d88-47cf-4416-b8f7-9c204bbd8f7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2efc99aac4564f538819203bbe5d8865'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:19.291657 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9c347d88-47cf-4416-b8f7-9c204bbd8f7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2efc99aac4564f538819203bbe5d8865'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:19.300111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9c347d88-47cf-4416-b8f7-9c204bbd8f7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2efc99aac4564f538819203bbe5d8865'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:19.300745 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9c347d88-47cf-4416-b8f7-9c204bbd8f7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2efc99aac4564f538819203bbe5d8865'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:19.309872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1044/2097] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:22:19 2022] DELETE /identity/v3/users/2efc99aac4564f538819203bbe5d8865 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:19.662705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e82f1-bd27-4acb-8a8c-44deaa804c28 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:19.664197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-aa5e82f1-bd27-4acb-8a8c-44deaa804c28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:19.678091 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e82f1-bd27-4acb-8a8c-44deaa804c28 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:19.744843 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2a99f-3cb0-4e93-b187-7b49152b1356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:19.745458 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2a99f-3cb0-4e93-b187-7b49152b1356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:19.745746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2a99f-3cb0-4e93-b187-7b49152b1356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:19.746020 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2a99f-3cb0-4e93-b187-7b49152b1356 admin admin] PATH_INFO: `/v3/projects/9259f222ca8f4eb5bb6a72415593f0e4` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:19.747231 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef2a99f-3cb0-4e93-b187-7b49152b1356 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9259f222ca8f4eb5bb6a72415593f0e4)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:19.749153 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef2a99f-3cb0-4e93-b187-7b49152b1356 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:19.797067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-fef2a99f-3cb0-4e93-b187-7b49152b1356 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9259f222ca8f4eb5bb6a72415593f0e4', 'request_id': 'req-fef2a99f-3cb0-4e93-b187-7b49152b1356'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:19.802723 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-fef2a99f-3cb0-4e93-b187-7b49152b1356 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9259f222ca8f4eb5bb6a72415593f0e4', 'request_id': 'req-fef2a99f-3cb0-4e93-b187-7b49152b1356'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:19.809994 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-fef2a99f-3cb0-4e93-b187-7b49152b1356 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9259f222ca8f4eb5bb6a72415593f0e4', 'request_id': 'req-fef2a99f-3cb0-4e93-b187-7b49152b1356'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:19.823523 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-fef2a99f-3cb0-4e93-b187-7b49152b1356 admin admin] The token cache is being invalidate because project 9259f222ca8f4eb5bb6a72415593f0e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:22:19.823848 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-fef2a99f-3cb0-4e93-b187-7b49152b1356 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:19.824477 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-fef2a99f-3cb0-4e93-b187-7b49152b1356 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:19.825966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1054/2098] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:22:19 2022] DELETE /identity/v3/projects/9259f222ca8f4eb5bb6a72415593f0e4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:19.831067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52964419-ba48-4394-9f5a-1cbab17ff825 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:19.831592 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52964419-ba48-4394-9f5a-1cbab17ff825 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:19.831592 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52964419-ba48-4394-9f5a-1cbab17ff825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:19.842341 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:19.854907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-52964419-ba48-4394-9f5a-1cbab17ff825 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:19.856128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-52964419-ba48-4394-9f5a-1cbab17ff825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:19.914433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1045/2099] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:19.922964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1adc53-2d44-4bce-806d-13666b767ee0 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:19.923278 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1adc53-2d44-4bce-806d-13666b767ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:19.923545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1adc53-2d44-4bce-806d-13666b767ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:19.943485 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-3e1adc53-2d44-4bce-806d-13666b767ee0 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:19.944667 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-3e1adc53-2d44-4bce-806d-13666b767ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:19.953071 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1055/2100] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:19.957308 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52964419-ba48-4394-9f5a-1cbab17ff825 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:19.967195 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52964419-ba48-4394-9f5a-1cbab17ff825 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:19.971811 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a28e4c-90d8-4fec-9b10-92ffeb410b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:19.972411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a28e4c-90d8-4fec-9b10-92ffeb410b7e admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:19.972698 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a28e4c-90d8-4fec-9b10-92ffeb410b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:19.972969 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a28e4c-90d8-4fec-9b10-92ffeb410b7e admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:19.973624 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a28e4c-90d8-4fec-9b10-92ffeb410b7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:22:19.973931 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a28e4c-90d8-4fec-9b10-92ffeb410b7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:19.975570 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a28e4c-90d8-4fec-9b10-92ffeb410b7e admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:19.988021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1046/2101] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:22:19.992528 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1adc53-2d44-4bce-806d-13666b767ee0 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:20.002362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1adc53-2d44-4bce-806d-13666b767ee0 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:20.006971 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06603fc-09e9-4400-85ee-eddb3e308481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:20.007591 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06603fc-09e9-4400-85ee-eddb3e308481 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:20.007867 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06603fc-09e9-4400-85ee-eddb3e308481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:20.008138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06603fc-09e9-4400-85ee-eddb3e308481 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:20.008801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06603fc-09e9-4400-85ee-eddb3e308481 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:20.010690 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06603fc-09e9-4400-85ee-eddb3e308481 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:20.037651 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1056/2102] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:22:19 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:22:20.042377 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a28e4c-90d8-4fec-9b10-92ffeb410b7e admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:20.052509 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a28e4c-90d8-4fec-9b10-92ffeb410b7e admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:20.109403 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d0f45-fbb9-42d6-91ef-a2a4ec37af6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:20.110005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d0f45-fbb9-42d6-91ef-a2a4ec37af6e admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:20.110285 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d0f45-fbb9-42d6-91ef-a2a4ec37af6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:20.110555 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d0f45-fbb9-42d6-91ef-a2a4ec37af6e admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:20.111217 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2d0f45-fbb9-42d6-91ef-a2a4ec37af6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:20.113127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2d0f45-fbb9-42d6-91ef-a2a4ec37af6e admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:20.134234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1047/2103] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:22:20 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:22:20.138524 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06603fc-09e9-4400-85ee-eddb3e308481 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:20.148915 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06603fc-09e9-4400-85ee-eddb3e308481 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:20.154025 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f0e2b7-999b-44d5-bc17-a08bdb87ea3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:20.154618 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0e2b7-999b-44d5-bc17-a08bdb87ea3f admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:20.154893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0e2b7-999b-44d5-bc17-a08bdb87ea3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:20.155205 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0e2b7-999b-44d5-bc17-a08bdb87ea3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:20.155851 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f0e2b7-999b-44d5-bc17-a08bdb87ea3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:20.157444 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f0e2b7-999b-44d5-bc17-a08bdb87ea3f admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:20.167891 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1057/2104] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:22:20.171747 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d0f45-fbb9-42d6-91ef-a2a4ec37af6e admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:20.182295 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d0f45-fbb9-42d6-91ef-a2a4ec37af6e admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:20.186907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e638c2b-0f3b-4987-b05f-51174b4fe692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:20.187531 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e638c2b-0f3b-4987-b05f-51174b4fe692 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:20.187808 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e638c2b-0f3b-4987-b05f-51174b4fe692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:20.188075 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e638c2b-0f3b-4987-b05f-51174b4fe692 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:20.188706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e638c2b-0f3b-4987-b05f-51174b4fe692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:20.190293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e638c2b-0f3b-4987-b05f-51174b4fe692 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:20.201806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1048/2105] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:22:20.205861 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f0e2b7-999b-44d5-bc17-a08bdb87ea3f admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:20.216941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f0e2b7-999b-44d5-bc17-a08bdb87ea3f admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:20.221648 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd6012-6ae1-41e7-b360-1d6c9c4e7bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:20.222298 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd6012-6ae1-41e7-b360-1d6c9c4e7bd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:20.222578 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd6012-6ae1-41e7-b360-1d6c9c4e7bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:20.222854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd6012-6ae1-41e7-b360-1d6c9c4e7bd3 admin admin] PATH_INFO: `/v3/projects/1d6d73e6ec984284a6f1d093c855d232/users/10f2165a9b3f4cd5a93c7a68ccb282f0/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:20.238221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabd6012-6ae1-41e7-b360-1d6c9c4e7bd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d6d73e6ec984284a6f1d093c855d232, user_id=10f2165a9b3f4cd5a93c7a68ccb282f0, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:20.240971 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabd6012-6ae1-41e7-b360-1d6c9c4e7bd3 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:20.254250 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1058/2106] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:22:20 2022] PUT /identity/v3/projects/1d6d73e6ec984284a6f1d093c855d232/users/10f2165a9b3f4cd5a93c7a68ccb282f0/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:20.266084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9773a98-d9fc-45de-8ff8-ebdfca0ef341 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:20.266373 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9773a98-d9fc-45de-8ff8-ebdfca0ef341 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:20.266649 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9773a98-d9fc-45de-8ff8-ebdfca0ef341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:20.295166 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-b9773a98-d9fc-45de-8ff8-ebdfca0ef341 None None] MFA Rules not processed for user `10f2165a9b3f4cd5a93c7a68ccb282f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:20.296409 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b9773a98-d9fc-45de-8ff8-ebdfca0ef341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:20.357166 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:20.361027 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1049/2107] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:20 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:20.365441 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd6012-6ae1-41e7-b360-1d6c9c4e7bd3 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:20.382819 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:20.429327 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd6012-6ae1-41e7-b360-1d6c9c4e7bd3 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:20.434251 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265715e-f56b-41fe-a757-942b40ca0a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:20.434857 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265715e-f56b-41fe-a757-942b40ca0a21 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:20.435172 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265715e-f56b-41fe-a757-942b40ca0a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:20.435446 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265715e-f56b-41fe-a757-942b40ca0a21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:20.436110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265715e-f56b-41fe-a757-942b40ca0a21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:20.438025 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265715e-f56b-41fe-a757-942b40ca0a21 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:20.464277 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1059/2108] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:22:20 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:22:20.468354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9773a98-d9fc-45de-8ff8-ebdfca0ef341 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:20.480988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9773a98-d9fc-45de-8ff8-ebdfca0ef341 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:20.539821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9ca93-bb4f-4f46-8ef7-4131c8dc446a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:20.540356 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9ca93-bb4f-4f46-8ef7-4131c8dc446a admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:20.540801 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9ca93-bb4f-4f46-8ef7-4131c8dc446a admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:20.541076 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9ca93-bb4f-4f46-8ef7-4131c8dc446a admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:20.541719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9ca93-bb4f-4f46-8ef7-4131c8dc446a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:20.543702 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9ca93-bb4f-4f46-8ef7-4131c8dc446a admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:20.573727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1050/2109] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:22:20 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:22:20.577525 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265715e-f56b-41fe-a757-942b40ca0a21 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:20.590208 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265715e-f56b-41fe-a757-942b40ca0a21 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:20.595025 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bda983-e265-4287-b0cc-5622a221e057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:20.595670 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bda983-e265-4287-b0cc-5622a221e057 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:20.595947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bda983-e265-4287-b0cc-5622a221e057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:20.596214 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bda983-e265-4287-b0cc-5622a221e057 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:20.596861 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bda983-e265-4287-b0cc-5622a221e057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:20.598492 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bda983-e265-4287-b0cc-5622a221e057 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:20.611858 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1060/2110] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:22:20.616227 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9ca93-bb4f-4f46-8ef7-4131c8dc446a admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:20.628326 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9ca93-bb4f-4f46-8ef7-4131c8dc446a admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:20.633203 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97df6e1-7d8f-4660-92be-585dac678c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:20.634687 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97df6e1-7d8f-4660-92be-585dac678c6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:20.634972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97df6e1-7d8f-4660-92be-585dac678c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:20.635290 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97df6e1-7d8f-4660-92be-585dac678c6a admin admin] PATH_INFO: `/v3/projects/4142ab67282a48fc806f04b1d53a86d1/users/ce6b434609e64b89ae26029f06900a1d/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:20.653049 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97df6e1-7d8f-4660-92be-585dac678c6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4142ab67282a48fc806f04b1d53a86d1, user_id=ce6b434609e64b89ae26029f06900a1d, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:20.655774 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97df6e1-7d8f-4660-92be-585dac678c6a admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:20.671401 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1051/2111] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:22:20 2022] PUT /identity/v3/projects/4142ab67282a48fc806f04b1d53a86d1/users/ce6b434609e64b89ae26029f06900a1d/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:22:20.675571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bda983-e265-4287-b0cc-5622a221e057 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:20.686693 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bda983-e265-4287-b0cc-5622a221e057 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:20.744773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2d469-b5d7-43ba-8fdc-d407c7a659c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:20.745396 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2d469-b5d7-43ba-8fdc-d407c7a659c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:20.745674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2d469-b5d7-43ba-8fdc-d407c7a659c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:20.745945 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2d469-b5d7-43ba-8fdc-d407c7a659c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:20.746596 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2d469-b5d7-43ba-8fdc-d407c7a659c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:20.748386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2d469-b5d7-43ba-8fdc-d407c7a659c8 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:20.759888 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1061/2112] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:22:20.764445 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97df6e1-7d8f-4660-92be-585dac678c6a admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:20.775790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97df6e1-7d8f-4660-92be-585dac678c6a admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:20.780404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4047de6c-8ed1-472d-930e-39945111fdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:20.781056 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4047de6c-8ed1-472d-930e-39945111fdd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:20.781332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4047de6c-8ed1-472d-930e-39945111fdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:20.781602 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4047de6c-8ed1-472d-930e-39945111fdd4 admin admin] PATH_INFO: `/v3/projects/4142ab67282a48fc806f04b1d53a86d1/users/ce6b434609e64b89ae26029f06900a1d/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:20.783995 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4047de6c-8ed1-472d-930e-39945111fdd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4142ab67282a48fc806f04b1d53a86d1, user_id=ce6b434609e64b89ae26029f06900a1d, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:20.786128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4047de6c-8ed1-472d-930e-39945111fdd4 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:20.797235 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1052/2113] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:22:20 2022] PUT /identity/v3/projects/4142ab67282a48fc806f04b1d53a86d1/users/ce6b434609e64b89ae26029f06900a1d/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:20.807606 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66570be-39ce-4ed5-838a-bcfd1093f391 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:20.807896 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66570be-39ce-4ed5-838a-bcfd1093f391 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:20.808177 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66570be-39ce-4ed5-838a-bcfd1093f391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:20.837348 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-f66570be-39ce-4ed5-838a-bcfd1093f391 None None] MFA Rules not processed for user `ce6b434609e64b89ae26029f06900a1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:20.838625 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f66570be-39ce-4ed5-838a-bcfd1093f391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:20.911546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1062/2114] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:20 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:20.921961 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4047de6c-8ed1-472d-930e-39945111fdd4 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:20.923718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-4047de6c-8ed1-472d-930e-39945111fdd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:20.982779 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4047de6c-8ed1-472d-930e-39945111fdd4 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:20.986414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04306381-a863-4c7c-b34b-f29b040be5fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:20.987020 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04306381-a863-4c7c-b34b-f29b040be5fa service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:20.987329 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04306381-a863-4c7c-b34b-f29b040be5fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:20.987600 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04306381-a863-4c7c-b34b-f29b040be5fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:20.998325 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04306381-a863-4c7c-b34b-f29b040be5fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:20.999827 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04306381-a863-4c7c-b34b-f29b040be5fa service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:21.010253 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1053/2115] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:22:20 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:22:21.144878 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66570be-39ce-4ed5-838a-bcfd1093f391 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:21.157428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:21.208958 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66570be-39ce-4ed5-838a-bcfd1093f391 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:21.214023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f0d5b-e8d9-4e2e-bded-b8b4eb82f4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:21.214650 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f0d5b-e8d9-4e2e-bded-b8b4eb82f4ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:21.214977 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f0d5b-e8d9-4e2e-bded-b8b4eb82f4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:21.215280 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f0d5b-e8d9-4e2e-bded-b8b4eb82f4ea admin admin] PATH_INFO: `/v3/users/10f2165a9b3f4cd5a93c7a68ccb282f0` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:21.216388 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f0d5b-e8d9-4e2e-bded-b8b4eb82f4ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=10f2165a9b3f4cd5a93c7a68ccb282f0)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:21.218330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f0d5b-e8d9-4e2e-bded-b8b4eb82f4ea admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:21.262767 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d10f0d5b-e8d9-4e2e-bded-b8b4eb82f4ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10f2165a9b3f4cd5a93c7a68ccb282f0'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:21.270089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d10f0d5b-e8d9-4e2e-bded-b8b4eb82f4ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10f2165a9b3f4cd5a93c7a68ccb282f0'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:21.277980 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d10f0d5b-e8d9-4e2e-bded-b8b4eb82f4ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10f2165a9b3f4cd5a93c7a68ccb282f0'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:21.278693 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d10f0d5b-e8d9-4e2e-bded-b8b4eb82f4ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10f2165a9b3f4cd5a93c7a68ccb282f0'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:21.279384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d10f0d5b-e8d9-4e2e-bded-b8b4eb82f4ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10f2165a9b3f4cd5a93c7a68ccb282f0'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:21.296625 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1063/2116] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:22:21 2022] DELETE /identity/v3/users/10f2165a9b3f4cd5a93c7a68ccb282f0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:21.301186 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e0a38-d787-4223-9238-99532b3d19da None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:21.301415 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e0a38-d787-4223-9238-99532b3d19da None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:21.301689 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e0a38-d787-4223-9238-99532b3d19da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:21.322475 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-d68e0a38-d787-4223-9238-99532b3d19da None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:21.323704 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-d68e0a38-d787-4223-9238-99532b3d19da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:21.370750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:21.384502 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1054/2117] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:21.395068 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f0d5b-e8d9-4e2e-bded-b8b4eb82f4ea admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:21.396558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-d10f0d5b-e8d9-4e2e-bded-b8b4eb82f4ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:21.462259 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f0d5b-e8d9-4e2e-bded-b8b4eb82f4ea admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:21.466002 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee941314-6427-417d-b62e-da0ccada88cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:21.466606 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee941314-6427-417d-b62e-da0ccada88cd service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:21.466888 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee941314-6427-417d-b62e-da0ccada88cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:21.467196 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee941314-6427-417d-b62e-da0ccada88cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:21.476267 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee941314-6427-417d-b62e-da0ccada88cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:21.478272 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee941314-6427-417d-b62e-da0ccada88cd service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:21.488202 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1064/2118] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:22:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:22:21.759687 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e0a38-d787-4223-9238-99532b3d19da None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:21.761260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-d68e0a38-d787-4223-9238-99532b3d19da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:21.776338 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e0a38-d787-4223-9238-99532b3d19da None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:21.781276 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c39894-b943-498c-b167-d4d10bd61f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:21.781925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c39894-b943-498c-b167-d4d10bd61f6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:21.782202 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c39894-b943-498c-b167-d4d10bd61f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:21.782647 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c39894-b943-498c-b167-d4d10bd61f6d admin admin] PATH_INFO: `/v3/projects/1d6d73e6ec984284a6f1d093c855d232` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:21.783854 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c39894-b943-498c-b167-d4d10bd61f6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d6d73e6ec984284a6f1d093c855d232)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:21.786043 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c39894-b943-498c-b167-d4d10bd61f6d admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:21.832989 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-46c39894-b943-498c-b167-d4d10bd61f6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d6d73e6ec984284a6f1d093c855d232', 'request_id': 'req-46c39894-b943-498c-b167-d4d10bd61f6d'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:21.840747 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-46c39894-b943-498c-b167-d4d10bd61f6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d6d73e6ec984284a6f1d093c855d232', 'request_id': 'req-46c39894-b943-498c-b167-d4d10bd61f6d'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:21.846488 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-46c39894-b943-498c-b167-d4d10bd61f6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d6d73e6ec984284a6f1d093c855d232', 'request_id': 'req-46c39894-b943-498c-b167-d4d10bd61f6d'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:21.858340 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-46c39894-b943-498c-b167-d4d10bd61f6d admin admin] The token cache is being invalidate because project 1d6d73e6ec984284a6f1d093c855d232 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:22:21.858629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-46c39894-b943-498c-b167-d4d10bd61f6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:21.859291 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-46c39894-b943-498c-b167-d4d10bd61f6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:21.860783 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1055/2119] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:22:21 2022] DELETE /identity/v3/projects/1d6d73e6ec984284a6f1d093c855d232 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:21.864768 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee941314-6427-417d-b62e-da0ccada88cd service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:21.874489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee941314-6427-417d-b62e-da0ccada88cd service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:21.908946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:21.939501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e78c7-4eb9-4197-a812-f4f5c2861d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:21.940239 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e78c7-4eb9-4197-a812-f4f5c2861d9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:21.940601 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e78c7-4eb9-4197-a812-f4f5c2861d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:21.940949 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e78c7-4eb9-4197-a812-f4f5c2861d9a admin admin] PATH_INFO: `/v3/users/ce6b434609e64b89ae26029f06900a1d` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:21.942193 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5e78c7-4eb9-4197-a812-f4f5c2861d9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce6b434609e64b89ae26029f06900a1d)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:21.944392 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5e78c7-4eb9-4197-a812-f4f5c2861d9a admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:21.999635 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8e5e78c7-4eb9-4197-a812-f4f5c2861d9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce6b434609e64b89ae26029f06900a1d'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:22.007824 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8e5e78c7-4eb9-4197-a812-f4f5c2861d9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce6b434609e64b89ae26029f06900a1d'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:22.016400 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8e5e78c7-4eb9-4197-a812-f4f5c2861d9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce6b434609e64b89ae26029f06900a1d'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:22.017130 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8e5e78c7-4eb9-4197-a812-f4f5c2861d9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce6b434609e64b89ae26029f06900a1d'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:22.017804 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8e5e78c7-4eb9-4197-a812-f4f5c2861d9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce6b434609e64b89ae26029f06900a1d'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:22.037038 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1065/2120] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:22:21 2022] DELETE /identity/v3/users/ce6b434609e64b89ae26029f06900a1d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:22.302273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c39894-b943-498c-b167-d4d10bd61f6d admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:22.303787 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-46c39894-b943-498c-b167-d4d10bd61f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:22.317912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c39894-b943-498c-b167-d4d10bd61f6d admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:22.376183 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f154b-bed5-4738-9a1f-3a73a03b6bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:22.376742 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f154b-bed5-4738-9a1f-3a73a03b6bc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:22.377023 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f154b-bed5-4738-9a1f-3a73a03b6bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:22.377293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f154b-bed5-4738-9a1f-3a73a03b6bc1 admin admin] PATH_INFO: `/v3/projects/4142ab67282a48fc806f04b1d53a86d1` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:22.378414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6f154b-bed5-4738-9a1f-3a73a03b6bc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4142ab67282a48fc806f04b1d53a86d1)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:22.380328 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6f154b-bed5-4738-9a1f-3a73a03b6bc1 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:22.428355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-7c6f154b-bed5-4738-9a1f-3a73a03b6bc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4142ab67282a48fc806f04b1d53a86d1', 'request_id': 'req-7c6f154b-bed5-4738-9a1f-3a73a03b6bc1'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:22.435753 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-7c6f154b-bed5-4738-9a1f-3a73a03b6bc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4142ab67282a48fc806f04b1d53a86d1', 'request_id': 'req-7c6f154b-bed5-4738-9a1f-3a73a03b6bc1'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:22.441349 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-7c6f154b-bed5-4738-9a1f-3a73a03b6bc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4142ab67282a48fc806f04b1d53a86d1', 'request_id': 'req-7c6f154b-bed5-4738-9a1f-3a73a03b6bc1'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:22.452654 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-7c6f154b-bed5-4738-9a1f-3a73a03b6bc1 admin admin] The token cache is being invalidate because project 4142ab67282a48fc806f04b1d53a86d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:22:22.452935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-7c6f154b-bed5-4738-9a1f-3a73a03b6bc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:22.453581 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-7c6f154b-bed5-4738-9a1f-3a73a03b6bc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:22.455054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1056/2121] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:22:22 2022] DELETE /identity/v3/projects/4142ab67282a48fc806f04b1d53a86d1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:22.459980 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76a954-6d55-4fca-9c63-49232833f23a None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:22.460269 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76a954-6d55-4fca-9c63-49232833f23a None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:22.460548 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76a954-6d55-4fca-9c63-49232833f23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:22.481241 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-ea76a954-6d55-4fca-9c63-49232833f23a None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:22.482437 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-ea76a954-6d55-4fca-9c63-49232833f23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:22.538938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1066/2122] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:22.547258 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64249b5c-e93c-4884-82ec-ee0d5d54d41c None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:22.547550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64249b5c-e93c-4884-82ec-ee0d5d54d41c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:22.547829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64249b5c-e93c-4884-82ec-ee0d5d54d41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:22.567618 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-64249b5c-e93c-4884-82ec-ee0d5d54d41c None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:22.568817 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-64249b5c-e93c-4884-82ec-ee0d5d54d41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:22.577480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1057/2123] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:22.581607 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76a954-6d55-4fca-9c63-49232833f23a None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:22.592186 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76a954-6d55-4fca-9c63-49232833f23a None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:22.596783 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c6554-f682-46bc-b1e6-605a01d09dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:22.597382 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c6554-f682-46bc-b1e6-605a01d09dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:22.597662 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c6554-f682-46bc-b1e6-605a01d09dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:22.597940 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c6554-f682-46bc-b1e6-605a01d09dc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:22.598588 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7c6554-f682-46bc-b1e6-605a01d09dc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:22:22.598896 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7c6554-f682-46bc-b1e6-605a01d09dc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:22.600534 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7c6554-f682-46bc-b1e6-605a01d09dc6 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:22.618767 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1067/2124] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:22:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:22:22.619980 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64249b5c-e93c-4884-82ec-ee0d5d54d41c None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:22.624522 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:22.630824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64249b5c-e93c-4884-82ec-ee0d5d54d41c None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:22.638721 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e558a10-7e56-4671-9741-c3b11a9409fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:22.639356 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e558a10-7e56-4671-9741-c3b11a9409fd admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:22.639640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e558a10-7e56-4671-9741-c3b11a9409fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:22.639912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e558a10-7e56-4671-9741-c3b11a9409fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:22.640541 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e558a10-7e56-4671-9741-c3b11a9409fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:22.642454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e558a10-7e56-4671-9741-c3b11a9409fd admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:22.669769 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1058/2125] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:22:22 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:22:22.674049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c6554-f682-46bc-b1e6-605a01d09dc6 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:22.684674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c6554-f682-46bc-b1e6-605a01d09dc6 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:22.722366 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:22.745140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1a1bc-f1a7-4a6a-b4c0-51f60746ff41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:22.745743 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1a1bc-f1a7-4a6a-b4c0-51f60746ff41 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:22.746019 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1a1bc-f1a7-4a6a-b4c0-51f60746ff41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:22.746294 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1a1bc-f1a7-4a6a-b4c0-51f60746ff41 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:22.746930 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb1a1bc-f1a7-4a6a-b4c0-51f60746ff41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:22.748987 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb1a1bc-f1a7-4a6a-b4c0-51f60746ff41 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:22.770411 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1068/2126] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:22:22 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:22:22.775421 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e558a10-7e56-4671-9741-c3b11a9409fd admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:22.786022 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e558a10-7e56-4671-9741-c3b11a9409fd admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:22.790826 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc61ed9-e798-4bf4-b477-a73110b434c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:22.791455 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc61ed9-e798-4bf4-b477-a73110b434c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:22.791737 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc61ed9-e798-4bf4-b477-a73110b434c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:22.792008 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc61ed9-e798-4bf4-b477-a73110b434c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:22.792653 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc61ed9-e798-4bf4-b477-a73110b434c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:22.794235 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc61ed9-e798-4bf4-b477-a73110b434c6 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:22.804020 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1059/2127] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:22:22.808402 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1a1bc-f1a7-4a6a-b4c0-51f60746ff41 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:22.820623 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1a1bc-f1a7-4a6a-b4c0-51f60746ff41 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:22.827078 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5176ad1c-68e7-40d6-a34d-91103f3c02c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:22.827680 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176ad1c-68e7-40d6-a34d-91103f3c02c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:22.827955 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176ad1c-68e7-40d6-a34d-91103f3c02c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:22.828224 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176ad1c-68e7-40d6-a34d-91103f3c02c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:22.828859 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5176ad1c-68e7-40d6-a34d-91103f3c02c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:22.830445 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5176ad1c-68e7-40d6-a34d-91103f3c02c7 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:22.845074 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1069/2128] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:22:22.849263 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc61ed9-e798-4bf4-b477-a73110b434c6 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:22.860144 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc61ed9-e798-4bf4-b477-a73110b434c6 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:22.864786 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f874ebc-2b19-4800-b727-1b004cec3f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:22.865440 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f874ebc-2b19-4800-b727-1b004cec3f20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:22.865723 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f874ebc-2b19-4800-b727-1b004cec3f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:22.866002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f874ebc-2b19-4800-b727-1b004cec3f20 admin admin] PATH_INFO: `/v3/projects/448633e3022345599242a0ae115afe02/users/ab21a4bf3bac443281b8e54059daf14e/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:22.880670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f874ebc-2b19-4800-b727-1b004cec3f20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=448633e3022345599242a0ae115afe02, user_id=ab21a4bf3bac443281b8e54059daf14e, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:22.883307 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f874ebc-2b19-4800-b727-1b004cec3f20 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:22.897073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1060/2129] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:22:22 2022] PUT /identity/v3/projects/448633e3022345599242a0ae115afe02/users/ab21a4bf3bac443281b8e54059daf14e/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:22.908739 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fbcac-291f-4f98-861d-3346c7d8c2a1 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:22.909210 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fbcac-291f-4f98-861d-3346c7d8c2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:22.909348 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fbcac-291f-4f98-861d-3346c7d8c2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:22.937765 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-953fbcac-291f-4f98-861d-3346c7d8c2a1 None None] MFA Rules not processed for user `ab21a4bf3bac443281b8e54059daf14e`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:22.939014 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-953fbcac-291f-4f98-861d-3346c7d8c2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:23.008016 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1070/2130] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:23.013760 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f874ebc-2b19-4800-b727-1b004cec3f20 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:23.082239 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f874ebc-2b19-4800-b727-1b004cec3f20 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:23.089189 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc93ea4-047f-4530-ba6f-0e0e8272775c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:23.090026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc93ea4-047f-4530-ba6f-0e0e8272775c admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:23.090299 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc93ea4-047f-4530-ba6f-0e0e8272775c admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:23.090485 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc93ea4-047f-4530-ba6f-0e0e8272775c admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:23.091210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc93ea4-047f-4530-ba6f-0e0e8272775c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:23.093125 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc93ea4-047f-4530-ba6f-0e0e8272775c admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:23.123629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1061/2131] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:22:23 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:22:23.127663 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953fbcac-291f-4f98-861d-3346c7d8c2a1 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:23.142082 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953fbcac-291f-4f98-861d-3346c7d8c2a1 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:23.211203 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de0b9aa-630b-4293-8d6e-5563f0fe2a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:23.211881 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0b9aa-630b-4293-8d6e-5563f0fe2a0b admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:23.212215 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0b9aa-630b-4293-8d6e-5563f0fe2a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:23.212491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0b9aa-630b-4293-8d6e-5563f0fe2a0b admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:23.213324 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de0b9aa-630b-4293-8d6e-5563f0fe2a0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:23.215909 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de0b9aa-630b-4293-8d6e-5563f0fe2a0b admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:23.243387 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1071/2132] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:22:23 2022] POST /identity/v3/users => generated 297 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:22:23.247486 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc93ea4-047f-4530-ba6f-0e0e8272775c admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:23.258427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc93ea4-047f-4530-ba6f-0e0e8272775c admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:23.265970 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cce5eb-bb8d-4b8d-8c21-b521ff7ebfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:23.266607 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cce5eb-bb8d-4b8d-8c21-b521ff7ebfc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:23.266884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cce5eb-bb8d-4b8d-8c21-b521ff7ebfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:23.267185 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cce5eb-bb8d-4b8d-8c21-b521ff7ebfc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:23.267835 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cce5eb-bb8d-4b8d-8c21-b521ff7ebfc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:23.269426 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cce5eb-bb8d-4b8d-8c21-b521ff7ebfc0 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:23.282912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1062/2133] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:22:23.287125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de0b9aa-630b-4293-8d6e-5563f0fe2a0b admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:23.301523 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de0b9aa-630b-4293-8d6e-5563f0fe2a0b admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:23.309168 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cef2b-f981-4382-9ea5-c3d2765f294a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:23.310349 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cef2b-f981-4382-9ea5-c3d2765f294a admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:23.310696 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cef2b-f981-4382-9ea5-c3d2765f294a admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:23.310971 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cef2b-f981-4382-9ea5-c3d2765f294a admin admin] PATH_INFO: `/v3/projects/3f2f9907ed9b4eba8370bc61c8e91061/users/7e3124baab3849948ad88c4ae9c3fa95/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:23.333044 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3cef2b-f981-4382-9ea5-c3d2765f294a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f2f9907ed9b4eba8370bc61c8e91061, user_id=7e3124baab3849948ad88c4ae9c3fa95, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:23.335951 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3cef2b-f981-4382-9ea5-c3d2765f294a admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:23.349842 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1072/2134] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:22:23 2022] PUT /identity/v3/projects/3f2f9907ed9b4eba8370bc61c8e91061/users/7e3124baab3849948ad88c4ae9c3fa95/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:23.353573 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cce5eb-bb8d-4b8d-8c21-b521ff7ebfc0 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:23.363733 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cce5eb-bb8d-4b8d-8c21-b521ff7ebfc0 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:23.423876 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8affea30-7fe6-4f7e-a3e1-8385c4f24ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:23.424609 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8affea30-7fe6-4f7e-a3e1-8385c4f24ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:23.424897 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8affea30-7fe6-4f7e-a3e1-8385c4f24ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:23.425190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8affea30-7fe6-4f7e-a3e1-8385c4f24ac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:23.425831 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8affea30-7fe6-4f7e-a3e1-8385c4f24ac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:23.427441 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8affea30-7fe6-4f7e-a3e1-8385c4f24ac4 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:23.439359 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1063/2135] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:22:23.443512 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cef2b-f981-4382-9ea5-c3d2765f294a admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:23.454189 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cef2b-f981-4382-9ea5-c3d2765f294a admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:23.459238 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f45c1-f13f-479c-870c-9974013729c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:23.459888 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f45c1-f13f-479c-870c-9974013729c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:23.460176 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f45c1-f13f-479c-870c-9974013729c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:23.460484 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f45c1-f13f-479c-870c-9974013729c8 admin admin] PATH_INFO: `/v3/projects/3f2f9907ed9b4eba8370bc61c8e91061/users/7e3124baab3849948ad88c4ae9c3fa95/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:23.462522 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f45c1-f13f-479c-870c-9974013729c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f2f9907ed9b4eba8370bc61c8e91061, user_id=7e3124baab3849948ad88c4ae9c3fa95, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:23.465250 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f45c1-f13f-479c-870c-9974013729c8 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:23.471416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:23.476141 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1073/2136] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:22:23 2022] PUT /identity/v3/projects/3f2f9907ed9b4eba8370bc61c8e91061/users/7e3124baab3849948ad88c4ae9c3fa95/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:23.486765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39ce14-6444-40bb-a799-22c30a11426b None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:23.487064 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39ce14-6444-40bb-a799-22c30a11426b None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:23.487357 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39ce14-6444-40bb-a799-22c30a11426b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:23.515895 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-0f39ce14-6444-40bb-a799-22c30a11426b None None] MFA Rules not processed for user `7e3124baab3849948ad88c4ae9c3fa95`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:23.517095 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-0f39ce14-6444-40bb-a799-22c30a11426b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:23.520984 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:23.586379 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1064/2137] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:23.597016 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f45c1-f13f-479c-870c-9974013729c8 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:23.599338 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a63f45c1-f13f-479c-870c-9974013729c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:23.660047 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f45c1-f13f-479c-870c-9974013729c8 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:23.663924 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44102018-1bcd-4dc3-b216-953baeab9d92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:23.664536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44102018-1bcd-4dc3-b216-953baeab9d92 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:23.665317 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44102018-1bcd-4dc3-b216-953baeab9d92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:23.665590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44102018-1bcd-4dc3-b216-953baeab9d92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:23.675243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44102018-1bcd-4dc3-b216-953baeab9d92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:23.676684 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44102018-1bcd-4dc3-b216-953baeab9d92 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:23.688053 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1074/2138] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:22:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:22:24.084754 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39ce14-6444-40bb-a799-22c30a11426b None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:24.147892 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39ce14-6444-40bb-a799-22c30a11426b None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:24.153583 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f687c4-e5c5-457c-9a2c-0a0bcb48c3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:24.154208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f687c4-e5c5-457c-9a2c-0a0bcb48c3cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:24.154484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f687c4-e5c5-457c-9a2c-0a0bcb48c3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:24.154751 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f687c4-e5c5-457c-9a2c-0a0bcb48c3cb admin admin] PATH_INFO: `/v3/users/ab21a4bf3bac443281b8e54059daf14e` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:24.155883 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f687c4-e5c5-457c-9a2c-0a0bcb48c3cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab21a4bf3bac443281b8e54059daf14e)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:24.157800 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f687c4-e5c5-457c-9a2c-0a0bcb48c3cb admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:24.207699 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f1f687c4-e5c5-457c-9a2c-0a0bcb48c3cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab21a4bf3bac443281b8e54059daf14e'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:24.220607 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f1f687c4-e5c5-457c-9a2c-0a0bcb48c3cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab21a4bf3bac443281b8e54059daf14e'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:24.227923 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f1f687c4-e5c5-457c-9a2c-0a0bcb48c3cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab21a4bf3bac443281b8e54059daf14e'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:24.235880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f1f687c4-e5c5-457c-9a2c-0a0bcb48c3cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab21a4bf3bac443281b8e54059daf14e'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:24.236632 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f1f687c4-e5c5-457c-9a2c-0a0bcb48c3cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab21a4bf3bac443281b8e54059daf14e'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:24.242594 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1065/2139] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:22:24 2022] DELETE /identity/v3/users/ab21a4bf3bac443281b8e54059daf14e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:24.247164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3350f7-6992-404a-85ef-d91a8422acdc None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:24.247729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3350f7-6992-404a-85ef-d91a8422acdc None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:24.247729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3350f7-6992-404a-85ef-d91a8422acdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:24.268100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-3d3350f7-6992-404a-85ef-d91a8422acdc None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:24.269364 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-3d3350f7-6992-404a-85ef-d91a8422acdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:24.330807 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1075/2140] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:24.341214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f687c4-e5c5-457c-9a2c-0a0bcb48c3cb admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:24.342666 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f1f687c4-e5c5-457c-9a2c-0a0bcb48c3cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:24.366315 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:24.408761 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f687c4-e5c5-457c-9a2c-0a0bcb48c3cb admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:24.412640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac21c8a-c025-44ad-8a86-e0c9e85058e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:24.413259 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac21c8a-c025-44ad-8a86-e0c9e85058e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:24.413543 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac21c8a-c025-44ad-8a86-e0c9e85058e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:24.413814 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac21c8a-c025-44ad-8a86-e0c9e85058e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:24.423686 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac21c8a-c025-44ad-8a86-e0c9e85058e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:24.425227 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac21c8a-c025-44ad-8a86-e0c9e85058e4 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:24.435296 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1066/2141] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:22:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:22:24.711124 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3350f7-6992-404a-85ef-d91a8422acdc None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:24.712899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-3d3350f7-6992-404a-85ef-d91a8422acdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:24.725074 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3350f7-6992-404a-85ef-d91a8422acdc None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:24.730199 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25910611-55f7-4335-bc7b-16184e607fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:24.730816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25910611-55f7-4335-bc7b-16184e607fa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:24.731131 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25910611-55f7-4335-bc7b-16184e607fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:24.731407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25910611-55f7-4335-bc7b-16184e607fa7 admin admin] PATH_INFO: `/v3/projects/448633e3022345599242a0ae115afe02` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:24.732549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25910611-55f7-4335-bc7b-16184e607fa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=448633e3022345599242a0ae115afe02)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:24.734570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25910611-55f7-4335-bc7b-16184e607fa7 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:24.781771 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-25910611-55f7-4335-bc7b-16184e607fa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '448633e3022345599242a0ae115afe02', 'request_id': 'req-25910611-55f7-4335-bc7b-16184e607fa7'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:24.787186 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-25910611-55f7-4335-bc7b-16184e607fa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '448633e3022345599242a0ae115afe02', 'request_id': 'req-25910611-55f7-4335-bc7b-16184e607fa7'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:24.793496 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-25910611-55f7-4335-bc7b-16184e607fa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '448633e3022345599242a0ae115afe02', 'request_id': 'req-25910611-55f7-4335-bc7b-16184e607fa7'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:24.806206 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-25910611-55f7-4335-bc7b-16184e607fa7 admin admin] The token cache is being invalidate because project 448633e3022345599242a0ae115afe02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:22:24.806490 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-25910611-55f7-4335-bc7b-16184e607fa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:24.807146 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-25910611-55f7-4335-bc7b-16184e607fa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:24.808613 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1076/2142] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:22:24 2022] DELETE /identity/v3/projects/448633e3022345599242a0ae115afe02 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:24.812523 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac21c8a-c025-44ad-8a86-e0c9e85058e4 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:24.823369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac21c8a-c025-44ad-8a86-e0c9e85058e4 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:24.881727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f3360-d98a-459b-89a6-90dc28edb160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:24.882367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f3360-d98a-459b-89a6-90dc28edb160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:24.882650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f3360-d98a-459b-89a6-90dc28edb160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:24.882926 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f3360-d98a-459b-89a6-90dc28edb160 admin admin] PATH_INFO: `/v3/users/7e3124baab3849948ad88c4ae9c3fa95` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:24.885098 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f3360-d98a-459b-89a6-90dc28edb160 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e3124baab3849948ad88c4ae9c3fa95)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:24.887050 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f3360-d98a-459b-89a6-90dc28edb160 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:24.935135 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-282f3360-d98a-459b-89a6-90dc28edb160 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e3124baab3849948ad88c4ae9c3fa95'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:24.947763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-282f3360-d98a-459b-89a6-90dc28edb160 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e3124baab3849948ad88c4ae9c3fa95'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:24.954470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-282f3360-d98a-459b-89a6-90dc28edb160 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e3124baab3849948ad88c4ae9c3fa95'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:24.961954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-282f3360-d98a-459b-89a6-90dc28edb160 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e3124baab3849948ad88c4ae9c3fa95'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:24.962500 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-282f3360-d98a-459b-89a6-90dc28edb160 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e3124baab3849948ad88c4ae9c3fa95'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:24.968272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1067/2143] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:22:24 2022] DELETE /identity/v3/users/7e3124baab3849948ad88c4ae9c3fa95 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:25.235120 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25910611-55f7-4335-bc7b-16184e607fa7 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:25.236560 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-25910611-55f7-4335-bc7b-16184e607fa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:25.253161 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25910611-55f7-4335-bc7b-16184e607fa7 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:25.278265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:25.314290 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797ac05-d1ea-45d8-b993-5e91aea1d541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:25.314921 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797ac05-d1ea-45d8-b993-5e91aea1d541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:25.315238 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797ac05-d1ea-45d8-b993-5e91aea1d541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:25.315538 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797ac05-d1ea-45d8-b993-5e91aea1d541 admin admin] PATH_INFO: `/v3/projects/3f2f9907ed9b4eba8370bc61c8e91061` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:25.316669 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797ac05-d1ea-45d8-b993-5e91aea1d541 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f2f9907ed9b4eba8370bc61c8e91061)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:25.318614 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797ac05-d1ea-45d8-b993-5e91aea1d541 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:25.363893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c797ac05-d1ea-45d8-b993-5e91aea1d541 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f2f9907ed9b4eba8370bc61c8e91061', 'request_id': 'req-c797ac05-d1ea-45d8-b993-5e91aea1d541'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:25.370033 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c797ac05-d1ea-45d8-b993-5e91aea1d541 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f2f9907ed9b4eba8370bc61c8e91061', 'request_id': 'req-c797ac05-d1ea-45d8-b993-5e91aea1d541'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:25.376080 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c797ac05-d1ea-45d8-b993-5e91aea1d541 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f2f9907ed9b4eba8370bc61c8e91061', 'request_id': 'req-c797ac05-d1ea-45d8-b993-5e91aea1d541'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:25.389265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c797ac05-d1ea-45d8-b993-5e91aea1d541 admin admin] The token cache is being invalidate because project 3f2f9907ed9b4eba8370bc61c8e91061 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:22:25.389564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c797ac05-d1ea-45d8-b993-5e91aea1d541 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:25.390228 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c797ac05-d1ea-45d8-b993-5e91aea1d541 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:25.391770 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1077/2144] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:22:25 2022] DELETE /identity/v3/projects/3f2f9907ed9b4eba8370bc61c8e91061 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:25.398337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e5bbc-e8b6-485c-aec7-516a9139c629 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:25.398856 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e5bbc-e8b6-485c-aec7-516a9139c629 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:25.399789 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e5bbc-e8b6-485c-aec7-516a9139c629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:25.420869 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-b97e5bbc-e8b6-485c-aec7-516a9139c629 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:25.422057 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b97e5bbc-e8b6-485c-aec7-516a9139c629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:25.459918 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:25.486569 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1068/2145] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:25.495544 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8e1e9-85fa-4107-9b66-01d6c49ff1d3 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:25.495830 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8e1e9-85fa-4107-9b66-01d6c49ff1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:25.496109 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8e1e9-85fa-4107-9b66-01d6c49ff1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:25.518345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-e2c8e1e9-85fa-4107-9b66-01d6c49ff1d3 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:25.519595 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e2c8e1e9-85fa-4107-9b66-01d6c49ff1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:25.528621 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1078/2146] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:25.532428 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e5bbc-e8b6-485c-aec7-516a9139c629 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:25.543431 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e5bbc-e8b6-485c-aec7-516a9139c629 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:25.548475 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f48d5-dd04-4b13-87cf-6b834b9e5dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:25.549091 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f48d5-dd04-4b13-87cf-6b834b9e5dbc admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:25.549406 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f48d5-dd04-4b13-87cf-6b834b9e5dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:25.549676 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f48d5-dd04-4b13-87cf-6b834b9e5dbc admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:25.550326 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925f48d5-dd04-4b13-87cf-6b834b9e5dbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:22:25.550660 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925f48d5-dd04-4b13-87cf-6b834b9e5dbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:25.552283 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925f48d5-dd04-4b13-87cf-6b834b9e5dbc admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:25.564832 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1069/2147] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:22:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:22:25.568990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8e1e9-85fa-4107-9b66-01d6c49ff1d3 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:25.578156 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8e1e9-85fa-4107-9b66-01d6c49ff1d3 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:25.583700 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6952218-f800-48c7-94ab-072217639c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:25.584314 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6952218-f800-48c7-94ab-072217639c3e admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:25.584592 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6952218-f800-48c7-94ab-072217639c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:25.584865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6952218-f800-48c7-94ab-072217639c3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:25.585508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6952218-f800-48c7-94ab-072217639c3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:25.588094 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6952218-f800-48c7-94ab-072217639c3e admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:25.623750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1079/2148] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:22:25 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:22:25.627795 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f48d5-dd04-4b13-87cf-6b834b9e5dbc admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:25.637985 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f48d5-dd04-4b13-87cf-6b834b9e5dbc admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:25.697805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a72b3c-7f51-497e-9325-7478596e8fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:25.698424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a72b3c-7f51-497e-9325-7478596e8fbc admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:25.698701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a72b3c-7f51-497e-9325-7478596e8fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:25.698973 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a72b3c-7f51-497e-9325-7478596e8fbc admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:25.699647 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a72b3c-7f51-497e-9325-7478596e8fbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:25.701567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a72b3c-7f51-497e-9325-7478596e8fbc admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:25.728227 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1070/2149] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:22:25 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:22:25.729524 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6952218-f800-48c7-94ab-072217639c3e admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:25.740375 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6952218-f800-48c7-94ab-072217639c3e admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:25.745250 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38c0af-278c-4729-a6a4-814cd79ef3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:25.745853 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38c0af-278c-4729-a6a4-814cd79ef3de admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:25.746134 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38c0af-278c-4729-a6a4-814cd79ef3de admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:25.746403 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38c0af-278c-4729-a6a4-814cd79ef3de admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:25.747055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d38c0af-278c-4729-a6a4-814cd79ef3de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:25.748656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d38c0af-278c-4729-a6a4-814cd79ef3de admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:25.760138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1080/2150] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:22:25.767625 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a72b3c-7f51-497e-9325-7478596e8fbc admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:25.779260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a72b3c-7f51-497e-9325-7478596e8fbc admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:25.786068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72b470-cdbf-43c7-b627-e9be0e2a6889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:25.786693 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72b470-cdbf-43c7-b627-e9be0e2a6889 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:25.786998 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72b470-cdbf-43c7-b627-e9be0e2a6889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:25.787312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72b470-cdbf-43c7-b627-e9be0e2a6889 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:25.787966 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72b470-cdbf-43c7-b627-e9be0e2a6889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:25.789549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72b470-cdbf-43c7-b627-e9be0e2a6889 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:25.803064 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1071/2151] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:22:25.805586 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38c0af-278c-4729-a6a4-814cd79ef3de admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:25.815802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38c0af-278c-4729-a6a4-814cd79ef3de admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:25.820596 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe3d3e-50c6-43af-8f99-b53a7cd33b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:25.821907 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe3d3e-50c6-43af-8f99-b53a7cd33b71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:25.822243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe3d3e-50c6-43af-8f99-b53a7cd33b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:25.822528 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe3d3e-50c6-43af-8f99-b53a7cd33b71 admin admin] PATH_INFO: `/v3/projects/f2b1884aba40455bb816e354f17f59f8/users/6b57e92b5f35449296e71bc0efd355a2/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:25.837326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe3d3e-50c6-43af-8f99-b53a7cd33b71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2b1884aba40455bb816e354f17f59f8, user_id=6b57e92b5f35449296e71bc0efd355a2, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:25.840031 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe3d3e-50c6-43af-8f99-b53a7cd33b71 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:25.853978 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1081/2152] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:22:25 2022] PUT /identity/v3/projects/f2b1884aba40455bb816e354f17f59f8/users/6b57e92b5f35449296e71bc0efd355a2/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:25.864393 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326b7ac-063a-415f-bfb0-b5b48828aa81 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:25.864685 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326b7ac-063a-415f-bfb0-b5b48828aa81 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:25.864960 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326b7ac-063a-415f-bfb0-b5b48828aa81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:25.897262 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-0326b7ac-063a-415f-bfb0-b5b48828aa81 None None] MFA Rules not processed for user `6b57e92b5f35449296e71bc0efd355a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:25.898465 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-0326b7ac-063a-415f-bfb0-b5b48828aa81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:25.965259 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1072/2153] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:25.976107 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe3d3e-50c6-43af-8f99-b53a7cd33b71 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:25.977860 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-d4fe3d3e-50c6-43af-8f99-b53a7cd33b71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:26.034652 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe3d3e-50c6-43af-8f99-b53a7cd33b71 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:26.038334 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c14be-013c-4c5c-9f7a-70c2283cebc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:26.038933 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c14be-013c-4c5c-9f7a-70c2283cebc2 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:26.039251 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c14be-013c-4c5c-9f7a-70c2283cebc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:26.039526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c14be-013c-4c5c-9f7a-70c2283cebc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:26.045755 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:26.047942 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c14be-013c-4c5c-9f7a-70c2283cebc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:26.049705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c14be-013c-4c5c-9f7a-70c2283cebc2 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:26.059378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1082/2154] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:22:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:22:35.961684 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=80,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:36.062171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=2 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:46.026293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0326b7ac-063a-415f-bfb0-b5b48828aa81 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:46.089719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0326b7ac-063a-415f-bfb0-b5b48828aa81 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:46.094311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a094d-65f1-4825-9a5a-8f8345acceda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:46.094870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a094d-65f1-4825-9a5a-8f8345acceda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:46.095173 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a094d-65f1-4825-9a5a-8f8345acceda admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:46.095441 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a094d-65f1-4825-9a5a-8f8345acceda admin admin] PATH_INFO: `/v3/users/6b57e92b5f35449296e71bc0efd355a2` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:46.096544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56a094d-65f1-4825-9a5a-8f8345acceda admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b57e92b5f35449296e71bc0efd355a2)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:46.098736 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56a094d-65f1-4825-9a5a-8f8345acceda admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:46.145328 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-c56a094d-65f1-4825-9a5a-8f8345acceda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b57e92b5f35449296e71bc0efd355a2'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:46.157365 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-c56a094d-65f1-4825-9a5a-8f8345acceda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b57e92b5f35449296e71bc0efd355a2'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:46.164088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-c56a094d-65f1-4825-9a5a-8f8345acceda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b57e92b5f35449296e71bc0efd355a2'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:46.172572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-c56a094d-65f1-4825-9a5a-8f8345acceda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b57e92b5f35449296e71bc0efd355a2'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:46.173233 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-c56a094d-65f1-4825-9a5a-8f8345acceda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b57e92b5f35449296e71bc0efd355a2'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:46.179160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1073/2155] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:22:46 2022] DELETE /identity/v3/users/6b57e92b5f35449296e71bc0efd355a2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:46.183826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a8b94-0e71-4661-bfd7-a8986546e067 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:46.184307 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a8b94-0e71-4661-bfd7-a8986546e067 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:46.184405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a8b94-0e71-4661-bfd7-a8986546e067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:46.204551 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-f66a8b94-0e71-4661-bfd7-a8986546e067 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:46.205753 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f66a8b94-0e71-4661-bfd7-a8986546e067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:46.270070 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1083/2156] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:46.280899 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a094d-65f1-4825-9a5a-8f8345acceda admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:46.282323 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c56a094d-65f1-4825-9a5a-8f8345acceda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:46.338613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a094d-65f1-4825-9a5a-8f8345acceda admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:46.343371 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7d651-d22a-4a91-b34a-60a37a54f0e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:46.343981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7d651-d22a-4a91-b34a-60a37a54f0e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:46.344262 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7d651-d22a-4a91-b34a-60a37a54f0e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:46.344590 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7d651-d22a-4a91-b34a-60a37a54f0e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:46.353643 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7d651-d22a-4a91-b34a-60a37a54f0e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:46.355101 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7d651-d22a-4a91-b34a-60a37a54f0e2 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:46.365203 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1074/2157] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:22:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:22:46.669131 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a8b94-0e71-4661-bfd7-a8986546e067 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:46.670868 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f66a8b94-0e71-4661-bfd7-a8986546e067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:46.697444 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a8b94-0e71-4661-bfd7-a8986546e067 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:46.704575 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ed18f-1ae0-4e23-b31e-3373b631941c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:46.704720 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7d651-d22a-4a91-b34a-60a37a54f0e2 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:46.705562 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ed18f-1ae0-4e23-b31e-3373b631941c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:46.705959 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ed18f-1ae0-4e23-b31e-3373b631941c admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:46.706115 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-25f7d651-d22a-4a91-b34a-60a37a54f0e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:46.706436 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ed18f-1ae0-4e23-b31e-3373b631941c admin admin] PATH_INFO: `/v3/projects/f2b1884aba40455bb816e354f17f59f8` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:46.707783 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ed18f-1ae0-4e23-b31e-3373b631941c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2b1884aba40455bb816e354f17f59f8)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:46.709793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ed18f-1ae0-4e23-b31e-3373b631941c admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:46.720279 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7d651-d22a-4a91-b34a-60a37a54f0e2 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:46.725533 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aec0e8-a84c-4463-bd53-964fe8664002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:46.726481 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aec0e8-a84c-4463-bd53-964fe8664002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:46.726481 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aec0e8-a84c-4463-bd53-964fe8664002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:46.726729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aec0e8-a84c-4463-bd53-964fe8664002 admin admin] PATH_INFO: `/v3/users/d98b14966ff0479bafea6b1d7c038581` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:46.727872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aec0e8-a84c-4463-bd53-964fe8664002 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d98b14966ff0479bafea6b1d7c038581)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:46.729818 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aec0e8-a84c-4463-bd53-964fe8664002 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:46.749886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:46.766318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c52ed18f-1ae0-4e23-b31e-3373b631941c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2b1884aba40455bb816e354f17f59f8', 'request_id': 'req-c52ed18f-1ae0-4e23-b31e-3373b631941c'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:46.773086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c52ed18f-1ae0-4e23-b31e-3373b631941c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2b1884aba40455bb816e354f17f59f8', 'request_id': 'req-c52ed18f-1ae0-4e23-b31e-3373b631941c'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:46.781298 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c52ed18f-1ae0-4e23-b31e-3373b631941c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2b1884aba40455bb816e354f17f59f8', 'request_id': 'req-c52ed18f-1ae0-4e23-b31e-3373b631941c'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:46.793897 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-16aec0e8-a84c-4463-bd53-964fe8664002 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd98b14966ff0479bafea6b1d7c038581'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:46.799330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c52ed18f-1ae0-4e23-b31e-3373b631941c admin admin] The token cache is being invalidate because project f2b1884aba40455bb816e354f17f59f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:22:46.799608 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c52ed18f-1ae0-4e23-b31e-3373b631941c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:46.800400 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c52ed18f-1ae0-4e23-b31e-3373b631941c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:46.802147 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1084/2158] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:22:46 2022] DELETE /identity/v3/projects/f2b1884aba40455bb816e354f17f59f8 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:46.812199 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-16aec0e8-a84c-4463-bd53-964fe8664002 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd98b14966ff0479bafea6b1d7c038581'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:46.821643 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-16aec0e8-a84c-4463-bd53-964fe8664002 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd98b14966ff0479bafea6b1d7c038581'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:46.831752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-16aec0e8-a84c-4463-bd53-964fe8664002 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd98b14966ff0479bafea6b1d7c038581'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:46.832539 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-16aec0e8-a84c-4463-bd53-964fe8664002 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd98b14966ff0479bafea6b1d7c038581'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:46.839798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1075/2159] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:22:46 2022] DELETE /identity/v3/users/d98b14966ff0479bafea6b1d7c038581 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:22:46.844663 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b845f8-564c-4471-bd8a-e99d79134546 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:46.844949 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b845f8-564c-4471-bd8a-e99d79134546 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:46.845221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b845f8-564c-4471-bd8a-e99d79134546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:46.871330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-a8b845f8-564c-4471-bd8a-e99d79134546 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:46.873633 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a8b845f8-564c-4471-bd8a-e99d79134546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:47.046368 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1085/2160] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:47.057034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aec0e8-a84c-4463-bd53-964fe8664002 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:47.058593 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-16aec0e8-a84c-4463-bd53-964fe8664002 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:47.125227 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aec0e8-a84c-4463-bd53-964fe8664002 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:47.129467 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79718a-6f54-4ef5-bcd8-9f3625fe770b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:47.130093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79718a-6f54-4ef5-bcd8-9f3625fe770b service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:47.130378 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79718a-6f54-4ef5-bcd8-9f3625fe770b service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:47.130654 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79718a-6f54-4ef5-bcd8-9f3625fe770b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:47.141469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79718a-6f54-4ef5-bcd8-9f3625fe770b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:47.142922 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79718a-6f54-4ef5-bcd8-9f3625fe770b service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:47.154964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1076/2161] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:22:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:22:47.492871 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b845f8-564c-4471-bd8a-e99d79134546 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:47.494695 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a8b845f8-564c-4471-bd8a-e99d79134546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:47.514624 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b845f8-564c-4471-bd8a-e99d79134546 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:47.520266 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a12867-267e-41f1-b1e3-06afdd9ba991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:47.521302 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a12867-267e-41f1-b1e3-06afdd9ba991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:47.521685 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a12867-267e-41f1-b1e3-06afdd9ba991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:47.522003 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a12867-267e-41f1-b1e3-06afdd9ba991 admin admin] PATH_INFO: `/v3/projects/1b066d1051854f6689c8c6c26ce82000` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:47.523242 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a12867-267e-41f1-b1e3-06afdd9ba991 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b066d1051854f6689c8c6c26ce82000)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:47.525343 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a12867-267e-41f1-b1e3-06afdd9ba991 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:47.550629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:47.579329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c5a12867-267e-41f1-b1e3-06afdd9ba991 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b066d1051854f6689c8c6c26ce82000', 'request_id': 'req-c5a12867-267e-41f1-b1e3-06afdd9ba991'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:47.585971 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c5a12867-267e-41f1-b1e3-06afdd9ba991 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b066d1051854f6689c8c6c26ce82000', 'request_id': 'req-c5a12867-267e-41f1-b1e3-06afdd9ba991'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:47.593834 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c5a12867-267e-41f1-b1e3-06afdd9ba991 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b066d1051854f6689c8c6c26ce82000', 'request_id': 'req-c5a12867-267e-41f1-b1e3-06afdd9ba991'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:47.608813 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c5a12867-267e-41f1-b1e3-06afdd9ba991 admin admin] The token cache is being invalidate because project 1b066d1051854f6689c8c6c26ce82000 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:22:47.609111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c5a12867-267e-41f1-b1e3-06afdd9ba991 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:47.609783 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c5a12867-267e-41f1-b1e3-06afdd9ba991 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:47.611563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1086/2162] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:22:47 2022] DELETE /identity/v3/projects/1b066d1051854f6689c8c6c26ce82000 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:47.616674 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea586e5d-5ed9-4e91-b2ed-d09d50a54d0a None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:47.616963 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea586e5d-5ed9-4e91-b2ed-d09d50a54d0a None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:47.617240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea586e5d-5ed9-4e91-b2ed-d09d50a54d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:47.640212 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-ea586e5d-5ed9-4e91-b2ed-d09d50a54d0a None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:47.641440 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-ea586e5d-5ed9-4e91-b2ed-d09d50a54d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:47.745078 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3117c-acd2-4df0-94c9-b60f64e76c6c None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:47.745495 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3117c-acd2-4df0-94c9-b60f64e76c6c None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:47.746310 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3117c-acd2-4df0-94c9-b60f64e76c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:47.759424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1077/2163] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:47.765952 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b269a-37cf-4745-9d08-e87b6e3508ad None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:47.766404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b269a-37cf-4745-9d08-e87b6e3508ad None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:47.766747 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b269a-37cf-4745-9d08-e87b6e3508ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:47.772574 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-48c3117c-acd2-4df0-94c9-b60f64e76c6c None None] MFA Rules not processed for user `95e5a1b739a54c3aa4dcb53e5bbe0452`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:47.774595 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-48c3117c-acd2-4df0-94c9-b60f64e76c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:47.791742 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-de4b269a-37cf-4745-9d08-e87b6e3508ad None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:47.793002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-de4b269a-37cf-4745-9d08-e87b6e3508ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:47.806163 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1078/2164] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:22:47.811134 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b269a-37cf-4745-9d08-e87b6e3508ad None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:47.823538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b269a-37cf-4745-9d08-e87b6e3508ad None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:47.830401 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f5822-e913-4d24-9d35-3b4e36fcba19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:47.831187 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f5822-e913-4d24-9d35-3b4e36fcba19 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:47.831572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f5822-e913-4d24-9d35-3b4e36fcba19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:47.831901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f5822-e913-4d24-9d35-3b4e36fcba19 admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:47.832623 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f5822-e913-4d24-9d35-3b4e36fcba19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:22:47.832992 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f5822-e913-4d24-9d35-3b4e36fcba19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:47.834750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f5822-e913-4d24-9d35-3b4e36fcba19 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:47.853983 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1079/2165] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:22:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:22:47.855581 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f5822-e913-4d24-9d35-3b4e36fcba19 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:47.859091 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1087/2166] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:47.866783 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f5822-e913-4d24-9d35-3b4e36fcba19 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:47.871608 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b168004e-465e-4e21-98e8-ed18da6a3f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:47.872214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168004e-465e-4e21-98e8-ed18da6a3f94 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:47.872490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168004e-465e-4e21-98e8-ed18da6a3f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:47.873821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168004e-465e-4e21-98e8-ed18da6a3f94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:47.874631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b168004e-465e-4e21-98e8-ed18da6a3f94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:47.876781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b168004e-465e-4e21-98e8-ed18da6a3f94 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:47.894276 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:47.914198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1080/2167] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:22:47 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:22:47.915075 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3117c-acd2-4df0-94c9-b60f64e76c6c None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:47.926422 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3117c-acd2-4df0-94c9-b60f64e76c6c None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:47.990237 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c30413-2638-4d62-88bd-307c1eff260f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:47.990850 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c30413-2638-4d62-88bd-307c1eff260f admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:47.991174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c30413-2638-4d62-88bd-307c1eff260f admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:47.991452 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c30413-2638-4d62-88bd-307c1eff260f admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:47.992194 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c30413-2638-4d62-88bd-307c1eff260f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:47.994153 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c30413-2638-4d62-88bd-307c1eff260f admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:48.017899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1088/2168] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:22:47 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:22:48.021781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b168004e-465e-4e21-98e8-ed18da6a3f94 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:48.035049 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b168004e-465e-4e21-98e8-ed18da6a3f94 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:48.040959 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b386e-370f-4167-8b5f-1af7c06ee261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:48.041711 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b386e-370f-4167-8b5f-1af7c06ee261 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:48.042106 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b386e-370f-4167-8b5f-1af7c06ee261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:48.042445 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b386e-370f-4167-8b5f-1af7c06ee261 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:48.043358 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b386e-370f-4167-8b5f-1af7c06ee261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:48.045063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b386e-370f-4167-8b5f-1af7c06ee261 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:48.067689 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1081/2169] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:22:48.068702 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c30413-2638-4d62-88bd-307c1eff260f admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:48.081676 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c30413-2638-4d62-88bd-307c1eff260f admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:48.086735 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e8995-f245-4ce6-8b4f-99eda02b7c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:48.087663 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e8995-f245-4ce6-8b4f-99eda02b7c7a admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:48.088012 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e8995-f245-4ce6-8b4f-99eda02b7c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:48.088351 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e8995-f245-4ce6-8b4f-99eda02b7c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:48.089077 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470e8995-f245-4ce6-8b4f-99eda02b7c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:48.090747 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470e8995-f245-4ce6-8b4f-99eda02b7c7a admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:48.101997 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1089/2170] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:22:48.106914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b386e-370f-4167-8b5f-1af7c06ee261 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:48.119548 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b386e-370f-4167-8b5f-1af7c06ee261 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:48.130646 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce4e42-5572-47a2-8c92-f470e40a4739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:48.132116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce4e42-5572-47a2-8c92-f470e40a4739 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:48.132503 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce4e42-5572-47a2-8c92-f470e40a4739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:48.132886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce4e42-5572-47a2-8c92-f470e40a4739 admin admin] PATH_INFO: `/v3/projects/d201c4183dde4ac693ac92c43c02df77/users/f9add734db2a4adc9857040166c02675/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:48.150495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce4e42-5572-47a2-8c92-f470e40a4739 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d201c4183dde4ac693ac92c43c02df77, user_id=f9add734db2a4adc9857040166c02675, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:48.153443 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce4e42-5572-47a2-8c92-f470e40a4739 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:48.182569 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1082/2171] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:22:48 2022] PUT /identity/v3/projects/d201c4183dde4ac693ac92c43c02df77/users/f9add734db2a4adc9857040166c02675/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:22:48.189611 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce044b-0440-460d-bd96-d8fb8dce6be3 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:48.189929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce044b-0440-460d-bd96-d8fb8dce6be3 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:48.190296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce044b-0440-460d-bd96-d8fb8dce6be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:48.213248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-69ce044b-0440-460d-bd96-d8fb8dce6be3 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:48.214774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-69ce044b-0440-460d-bd96-d8fb8dce6be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:48.245971 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:48.284348 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1090/2172] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:48.295107 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce4e42-5572-47a2-8c92-f470e40a4739 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:48.297423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-39ce4e42-5572-47a2-8c92-f470e40a4739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:48.367491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce4e42-5572-47a2-8c92-f470e40a4739 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:48.374797 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad7f64-3777-4912-bc2d-ce8330affb01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:48.375530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad7f64-3777-4912-bc2d-ce8330affb01 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:48.375938 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad7f64-3777-4912-bc2d-ce8330affb01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:48.376278 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad7f64-3777-4912-bc2d-ce8330affb01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:48.387143 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adad7f64-3777-4912-bc2d-ce8330affb01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:48.388596 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adad7f64-3777-4912-bc2d-ce8330affb01 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:48.401405 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1083/2173] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:22:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:22:54.051818 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509eab9-5b6d-452b-afbb-e86a5b346f0f None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:54.052356 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509eab9-5b6d-452b-afbb-e86a5b346f0f None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:54.052356 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509eab9-5b6d-452b-afbb-e86a5b346f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:54.098923 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-1509eab9-5b6d-452b-afbb-e86a5b346f0f None None] MFA Rules not processed for user `f9add734db2a4adc9857040166c02675`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:54.100185 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-1509eab9-5b6d-452b-afbb-e86a5b346f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:54.218868 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1091/2174] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:54.227366 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad7f64-3777-4912-bc2d-ce8330affb01 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:54.240132 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad7f64-3777-4912-bc2d-ce8330affb01 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:54.249356 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf899c5-7abc-4b72-91a9-d0617c4f78cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:54.250886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf899c5-7abc-4b72-91a9-d0617c4f78cb admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:54.252189 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf899c5-7abc-4b72-91a9-d0617c4f78cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:54.253238 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf899c5-7abc-4b72-91a9-d0617c4f78cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:54.254594 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf899c5-7abc-4b72-91a9-d0617c4f78cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:54.257920 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf899c5-7abc-4b72-91a9-d0617c4f78cb admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:54.307131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1084/2175] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:22:54 2022] POST /identity/v3/projects => generated 379 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:22:54.312886 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509eab9-5b6d-452b-afbb-e86a5b346f0f None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:54.328361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509eab9-5b6d-452b-afbb-e86a5b346f0f None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:54.424848 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bba62-7d0d-4346-9f80-fb4dc0ed56e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:54.425485 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bba62-7d0d-4346-9f80-fb4dc0ed56e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:54.425769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bba62-7d0d-4346-9f80-fb4dc0ed56e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:54.426049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bba62-7d0d-4346-9f80-fb4dc0ed56e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:54.426681 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462bba62-7d0d-4346-9f80-fb4dc0ed56e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:54.428682 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462bba62-7d0d-4346-9f80-fb4dc0ed56e7 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:54.474891 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1092/2176] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:22:54 2022] POST /identity/v3/users => generated 296 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:22:54.478569 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf899c5-7abc-4b72-91a9-d0617c4f78cb admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:54.491899 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf899c5-7abc-4b72-91a9-d0617c4f78cb admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:54.498972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13867175-45b2-4880-9d5d-d0ad897af4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:54.499508 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13867175-45b2-4880-9d5d-d0ad897af4de admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:54.500171 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13867175-45b2-4880-9d5d-d0ad897af4de admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:54.500451 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13867175-45b2-4880-9d5d-d0ad897af4de admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:54.501283 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13867175-45b2-4880-9d5d-d0ad897af4de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:54.502908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13867175-45b2-4880-9d5d-d0ad897af4de admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:54.523549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1085/2177] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:22:54.525061 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bba62-7d0d-4346-9f80-fb4dc0ed56e7 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:54.541791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bba62-7d0d-4346-9f80-fb4dc0ed56e7 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:54.555305 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22662a-6b4e-4082-bac7-0ccfcef2348c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:54.555998 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22662a-6b4e-4082-bac7-0ccfcef2348c admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:54.556277 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22662a-6b4e-4082-bac7-0ccfcef2348c admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:54.556546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22662a-6b4e-4082-bac7-0ccfcef2348c admin admin] PATH_INFO: `/v3/projects/e99f9b4e4d974334b90fcb122bdc2342/users/61ba3008e5284162996252a821103d9a/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:54.581818 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22662a-6b4e-4082-bac7-0ccfcef2348c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e99f9b4e4d974334b90fcb122bdc2342, user_id=61ba3008e5284162996252a821103d9a, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:54.584584 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22662a-6b4e-4082-bac7-0ccfcef2348c admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:54.602186 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1093/2178] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:22:54 2022] PUT /identity/v3/projects/e99f9b4e4d974334b90fcb122bdc2342/users/61ba3008e5284162996252a821103d9a/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:22:54.606182 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13867175-45b2-4880-9d5d-d0ad897af4de admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:54.616926 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13867175-45b2-4880-9d5d-d0ad897af4de admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:54.681314 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c28be-6f04-43e5-ae03-84175b91fbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:54.681928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c28be-6f04-43e5-ae03-84175b91fbac admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:54.682208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c28be-6f04-43e5-ae03-84175b91fbac admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:54.682483 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c28be-6f04-43e5-ae03-84175b91fbac admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:54.683189 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2c28be-6f04-43e5-ae03-84175b91fbac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:54.684820 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2c28be-6f04-43e5-ae03-84175b91fbac admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:54.696156 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1086/2179] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:22:54.700079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22662a-6b4e-4082-bac7-0ccfcef2348c admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:54.711402 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22662a-6b4e-4082-bac7-0ccfcef2348c admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:54.720215 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f1c3f-0a3f-43df-81b1-8adcb3f45461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:54.720884 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f1c3f-0a3f-43df-81b1-8adcb3f45461 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:54.721167 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f1c3f-0a3f-43df-81b1-8adcb3f45461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:54.721442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f1c3f-0a3f-43df-81b1-8adcb3f45461 admin admin] PATH_INFO: `/v3/projects/e99f9b4e4d974334b90fcb122bdc2342/users/61ba3008e5284162996252a821103d9a/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:54.723418 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f1c3f-0a3f-43df-81b1-8adcb3f45461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e99f9b4e4d974334b90fcb122bdc2342, user_id=61ba3008e5284162996252a821103d9a, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:54.726137 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f1c3f-0a3f-43df-81b1-8adcb3f45461 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:54.737990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1094/2180] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:22:54 2022] PUT /identity/v3/projects/e99f9b4e4d974334b90fcb122bdc2342/users/61ba3008e5284162996252a821103d9a/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:58.144393 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c28be-6f04-43e5-ae03-84175b91fbac admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:58.146313 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-de2c28be-6f04-43e5-ae03-84175b91fbac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:58.172151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c28be-6f04-43e5-ae03-84175b91fbac admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:58.239244 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:58.293563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299718de-58c6-4ebc-8ec7-b62543e8f739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:58.295324 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299718de-58c6-4ebc-8ec7-b62543e8f739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:58.297776 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299718de-58c6-4ebc-8ec7-b62543e8f739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:58.298200 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299718de-58c6-4ebc-8ec7-b62543e8f739 admin admin] PATH_INFO: `/v3/users/d10b50451d4b48f0b2377626ed609de4` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:58.300724 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299718de-58c6-4ebc-8ec7-b62543e8f739 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d10b50451d4b48f0b2377626ed609de4)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:58.303220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299718de-58c6-4ebc-8ec7-b62543e8f739 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:58.387743 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-299718de-58c6-4ebc-8ec7-b62543e8f739 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd10b50451d4b48f0b2377626ed609de4'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:58.412812 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-299718de-58c6-4ebc-8ec7-b62543e8f739 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd10b50451d4b48f0b2377626ed609de4'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:58.427541 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-299718de-58c6-4ebc-8ec7-b62543e8f739 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd10b50451d4b48f0b2377626ed609de4'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:58.438974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-299718de-58c6-4ebc-8ec7-b62543e8f739 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd10b50451d4b48f0b2377626ed609de4'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:58.440537 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-299718de-58c6-4ebc-8ec7-b62543e8f739 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd10b50451d4b48f0b2377626ed609de4'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:58.458428 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1087/2181] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:22:58 2022] DELETE /identity/v3/users/d10b50451d4b48f0b2377626ed609de4 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:58.795279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f1c3f-0a3f-43df-81b1-8adcb3f45461 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:58.798625 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4e7f1c3f-0a3f-43df-81b1-8adcb3f45461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:58.833232 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f1c3f-0a3f-43df-81b1-8adcb3f45461 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:58.859737 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:59.037433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74091be5-d7d8-401e-952f-6437bf1c1105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:59.040656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74091be5-d7d8-401e-952f-6437bf1c1105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:59.042078 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74091be5-d7d8-401e-952f-6437bf1c1105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:59.042373 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74091be5-d7d8-401e-952f-6437bf1c1105 admin admin] PATH_INFO: `/v3/projects/788926197c5042b99931f959426cff47` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:59.043777 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74091be5-d7d8-401e-952f-6437bf1c1105 admin admin] RBAC: Authorizing `identity:delete_project(project_id=788926197c5042b99931f959426cff47)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:59.045990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74091be5-d7d8-401e-952f-6437bf1c1105 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:59.121727 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-74091be5-d7d8-401e-952f-6437bf1c1105 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '788926197c5042b99931f959426cff47', 'request_id': 'req-74091be5-d7d8-401e-952f-6437bf1c1105'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:59.134421 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-74091be5-d7d8-401e-952f-6437bf1c1105 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '788926197c5042b99931f959426cff47', 'request_id': 'req-74091be5-d7d8-401e-952f-6437bf1c1105'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:59.144250 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-74091be5-d7d8-401e-952f-6437bf1c1105 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '788926197c5042b99931f959426cff47', 'request_id': 'req-74091be5-d7d8-401e-952f-6437bf1c1105'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:59.165346 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-74091be5-d7d8-401e-952f-6437bf1c1105 admin admin] The token cache is being invalidate because project 788926197c5042b99931f959426cff47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:22:59.165814 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-74091be5-d7d8-401e-952f-6437bf1c1105 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:59.166513 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-74091be5-d7d8-401e-952f-6437bf1c1105 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:22:59.168339 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1095/2182] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:22:58 2022] DELETE /identity/v3/projects/788926197c5042b99931f959426cff47 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:59.176829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceea775-251e-4d91-b4e4-41fcad517c67 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:59.177305 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceea775-251e-4d91-b4e4-41fcad517c67 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:59.177807 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceea775-251e-4d91-b4e4-41fcad517c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:59.216257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-eceea775-251e-4d91-b4e4-41fcad517c67 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:59.218716 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-eceea775-251e-4d91-b4e4-41fcad517c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:59.361174 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1088/2183] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:22:59.370090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad2524-cdf2-4ff2-af8c-53164239c9df None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:59.370090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad2524-cdf2-4ff2-af8c-53164239c9df None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:59.370286 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad2524-cdf2-4ff2-af8c-53164239c9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:59.418403 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-7fad2524-cdf2-4ff2-af8c-53164239c9df None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:59.421327 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-7fad2524-cdf2-4ff2-af8c-53164239c9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:22:59.432094 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1096/2184] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:22:59.436889 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceea775-251e-4d91-b4e4-41fcad517c67 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:59.446816 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceea775-251e-4d91-b4e4-41fcad517c67 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:59.454592 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ffe7f-b8c7-4084-8fd0-745513a142f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:59.456574 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ffe7f-b8c7-4084-8fd0-745513a142f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:59.457411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ffe7f-b8c7-4084-8fd0-745513a142f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:59.457823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ffe7f-b8c7-4084-8fd0-745513a142f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:59.458517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ffe7f-b8c7-4084-8fd0-745513a142f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:22:59.458828 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ffe7f-b8c7-4084-8fd0-745513a142f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:59.460657 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ffe7f-b8c7-4084-8fd0-745513a142f5 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:59.478888 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1089/2185] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:22:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:22:59.479783 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad2524-cdf2-4ff2-af8c-53164239c9df None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:59.490134 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad2524-cdf2-4ff2-af8c-53164239c9df None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:59.495082 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3bed1-1e03-419b-9a72-4617504c140a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:59.495724 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3bed1-1e03-419b-9a72-4617504c140a admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:59.496000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3bed1-1e03-419b-9a72-4617504c140a admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:59.496323 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3bed1-1e03-419b-9a72-4617504c140a admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:59.496973 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3bed1-1e03-419b-9a72-4617504c140a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:59.501031 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3bed1-1e03-419b-9a72-4617504c140a admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:59.536823 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1097/2186] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:22:59 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:22:59.540704 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ffe7f-b8c7-4084-8fd0-745513a142f5 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:59.552640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ffe7f-b8c7-4084-8fd0-745513a142f5 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:59.593842 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:22:59.632849 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824bed14-b02b-498f-b9ff-3b0a01b409e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:59.633487 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bed14-b02b-498f-b9ff-3b0a01b409e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:59.633767 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bed14-b02b-498f-b9ff-3b0a01b409e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:59.634921 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bed14-b02b-498f-b9ff-3b0a01b409e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:59.635924 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824bed14-b02b-498f-b9ff-3b0a01b409e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:59.639097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824bed14-b02b-498f-b9ff-3b0a01b409e3 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:59.667090 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1090/2187] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:22:59 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:22:59.671825 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3bed1-1e03-419b-9a72-4617504c140a admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:59.683328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3bed1-1e03-419b-9a72-4617504c140a admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:59.689979 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8bb79-f2a1-4bed-925b-1788c94bd527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:59.690605 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8bb79-f2a1-4bed-925b-1788c94bd527 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:59.690881 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8bb79-f2a1-4bed-925b-1788c94bd527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:59.691185 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8bb79-f2a1-4bed-925b-1788c94bd527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:59.691842 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8bb79-f2a1-4bed-925b-1788c94bd527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:59.693462 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8bb79-f2a1-4bed-925b-1788c94bd527 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:59.720318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1098/2188] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:22:59.721290 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824bed14-b02b-498f-b9ff-3b0a01b409e3 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:59.737274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824bed14-b02b-498f-b9ff-3b0a01b409e3 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:59.747413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238fbe27-d744-4790-8812-3626e238d5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:59.748627 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fbe27-d744-4790-8812-3626e238d5f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:59.748982 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fbe27-d744-4790-8812-3626e238d5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:59.749313 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fbe27-d744-4790-8812-3626e238d5f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:59.749969 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238fbe27-d744-4790-8812-3626e238d5f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:59.751917 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238fbe27-d744-4790-8812-3626e238d5f3 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:59.773529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1091/2189] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:22:59.776849 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8bb79-f2a1-4bed-925b-1788c94bd527 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:22:59.792714 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8bb79-f2a1-4bed-925b-1788c94bd527 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:22:59.799606 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659719ac-a095-4abf-99d4-5b2cb5fcaf2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:22:59.800308 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659719ac-a095-4abf-99d4-5b2cb5fcaf2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:59.800666 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659719ac-a095-4abf-99d4-5b2cb5fcaf2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:59.800951 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659719ac-a095-4abf-99d4-5b2cb5fcaf2a admin admin] PATH_INFO: `/v3/projects/c6173289fa144f04b0121f79a70db124/users/fa10b067020c43d2ae79aef38868f143/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:59.821697 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659719ac-a095-4abf-99d4-5b2cb5fcaf2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6173289fa144f04b0121f79a70db124, user_id=fa10b067020c43d2ae79aef38868f143, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:22:59.825345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659719ac-a095-4abf-99d4-5b2cb5fcaf2a admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:22:59.842954 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1099/2190] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:22:59 2022] PUT /identity/v3/projects/c6173289fa144f04b0121f79a70db124/users/fa10b067020c43d2ae79aef38868f143/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:22:59.854543 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7564b-ce91-43a0-a6bb-67078796a65e None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:22:59.854831 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7564b-ce91-43a0-a6bb-67078796a65e None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:22:59.855141 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7564b-ce91-43a0-a6bb-67078796a65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:22:59.888041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-34e7564b-ce91-43a0-a6bb-67078796a65e None None] MFA Rules not processed for user `fa10b067020c43d2ae79aef38868f143`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:22:59.890122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-34e7564b-ce91-43a0-a6bb-67078796a65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:23:00.007830 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1092/2191] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:22:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:23:00.012087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659719ac-a095-4abf-99d4-5b2cb5fcaf2a admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:23:00.062315 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:23:00.089930 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659719ac-a095-4abf-99d4-5b2cb5fcaf2a admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:23:00.095898 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05743485-6ece-4e8e-bc35-1405a642c3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:23:00.097411 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05743485-6ece-4e8e-bc35-1405a642c3b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:23:00.097699 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05743485-6ece-4e8e-bc35-1405a642c3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:23:00.097980 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05743485-6ece-4e8e-bc35-1405a642c3b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:23:00.098632 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05743485-6ece-4e8e-bc35-1405a642c3b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:23:00.105670 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05743485-6ece-4e8e-bc35-1405a642c3b8 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:23:00.143964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1100/2192] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:23:00 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:23:00.145460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7564b-ce91-43a0-a6bb-67078796a65e None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:23:00.160377 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7564b-ce91-43a0-a6bb-67078796a65e None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:23:00.226706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c05ba9-35b4-403b-9694-3faed23a01a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:23:00.227428 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c05ba9-35b4-403b-9694-3faed23a01a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:23:00.227726 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c05ba9-35b4-403b-9694-3faed23a01a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:23:00.227981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c05ba9-35b4-403b-9694-3faed23a01a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:23:00.228634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c05ba9-35b4-403b-9694-3faed23a01a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:23:00.230575 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c05ba9-35b4-403b-9694-3faed23a01a7 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:23:00.255027 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1093/2193] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:23:00 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:23:00.258671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05743485-6ece-4e8e-bc35-1405a642c3b8 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:23:00.270042 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05743485-6ece-4e8e-bc35-1405a642c3b8 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:23:00.275295 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd1c65-0389-4cda-9ff8-a283380f8fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:23:00.275998 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd1c65-0389-4cda-9ff8-a283380f8fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:23:00.276340 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd1c65-0389-4cda-9ff8-a283380f8fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:23:00.276820 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd1c65-0389-4cda-9ff8-a283380f8fa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:23:00.277540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd1c65-0389-4cda-9ff8-a283380f8fa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:23:00.279468 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd1c65-0389-4cda-9ff8-a283380f8fa8 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:23:00.294689 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1101/2194] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:23:00.296570 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c05ba9-35b4-403b-9694-3faed23a01a7 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:23:00.311959 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c05ba9-35b4-403b-9694-3faed23a01a7 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:23:00.319978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda4935-0922-4cfb-a7e7-097bc8436b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:23:00.321543 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda4935-0922-4cfb-a7e7-097bc8436b36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:23:00.321906 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda4935-0922-4cfb-a7e7-097bc8436b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:23:00.322797 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda4935-0922-4cfb-a7e7-097bc8436b36 admin admin] PATH_INFO: `/v3/projects/c32075d70bfb4c529e4a49c04f8c3fff/users/7d45e94776cb4bccaff824f761c13239/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:23:00.341074 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcda4935-0922-4cfb-a7e7-097bc8436b36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c32075d70bfb4c529e4a49c04f8c3fff, user_id=7d45e94776cb4bccaff824f761c13239, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:23:00.344548 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcda4935-0922-4cfb-a7e7-097bc8436b36 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:23:00.366029 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1094/2195] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:23:00 2022] PUT /identity/v3/projects/c32075d70bfb4c529e4a49c04f8c3fff/users/7d45e94776cb4bccaff824f761c13239/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:23:00.367301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd1c65-0389-4cda-9ff8-a283380f8fa8 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:23:00.379459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd1c65-0389-4cda-9ff8-a283380f8fa8 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:23:00.469767 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c562b86-793b-4cf2-8f1a-c9d6d98a6d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:23:00.470476 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c562b86-793b-4cf2-8f1a-c9d6d98a6d41 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:23:00.470830 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c562b86-793b-4cf2-8f1a-c9d6d98a6d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:23:00.471223 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c562b86-793b-4cf2-8f1a-c9d6d98a6d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:23:00.471955 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c562b86-793b-4cf2-8f1a-c9d6d98a6d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:23:00.473670 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c562b86-793b-4cf2-8f1a-c9d6d98a6d41 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:23:00.492234 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1102/2196] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:23:00.493522 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda4935-0922-4cfb-a7e7-097bc8436b36 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:23:00.515753 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda4935-0922-4cfb-a7e7-097bc8436b36 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:23:00.521227 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18e745-2739-49fc-9b71-020ab07fd256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:23:00.521956 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18e745-2739-49fc-9b71-020ab07fd256 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:23:00.523262 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18e745-2739-49fc-9b71-020ab07fd256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:23:00.523585 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18e745-2739-49fc-9b71-020ab07fd256 admin admin] PATH_INFO: `/v3/projects/c32075d70bfb4c529e4a49c04f8c3fff/users/7d45e94776cb4bccaff824f761c13239/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:23:00.531843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18e745-2739-49fc-9b71-020ab07fd256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c32075d70bfb4c529e4a49c04f8c3fff, user_id=7d45e94776cb4bccaff824f761c13239, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:23:00.534551 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18e745-2739-49fc-9b71-020ab07fd256 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:23:00.537143 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:23:00.555112 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1095/2197] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:23:00 2022] PUT /identity/v3/projects/c32075d70bfb4c529e4a49c04f8c3fff/users/7d45e94776cb4bccaff824f761c13239/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:23:00.562959 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0eec58-718a-43ba-b6d2-ff40ceb488c9 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:23:00.564181 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0eec58-718a-43ba-b6d2-ff40ceb488c9 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:23:00.564497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0eec58-718a-43ba-b6d2-ff40ceb488c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:23:00.603544 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-0a0eec58-718a-43ba-b6d2-ff40ceb488c9 None None] MFA Rules not processed for user `7d45e94776cb4bccaff824f761c13239`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:23:00.604920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-0a0eec58-718a-43ba-b6d2-ff40ceb488c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:23:00.710462 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1103/2198] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:23:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:23:00.725322 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18e745-2739-49fc-9b71-020ab07fd256 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:23:00.733630 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-8d18e745-2739-49fc-9b71-020ab07fd256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:23:00.812821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18e745-2739-49fc-9b71-020ab07fd256 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:23:00.816831 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366cbf2-534c-4bb9-993a-c36903e1863c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:23:00.817526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366cbf2-534c-4bb9-993a-c36903e1863c service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:23:00.817826 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366cbf2-534c-4bb9-993a-c36903e1863c service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:23:00.817997 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366cbf2-534c-4bb9-993a-c36903e1863c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:23:00.886944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d366cbf2-534c-4bb9-993a-c36903e1863c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:23:00.888448 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d366cbf2-534c-4bb9-993a-c36903e1863c service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:23:00.899981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1096/2199] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:23:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:23:01.652391 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf36eb-10e3-4d7b-9a98-06ca5dbd0932 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:23:01.652905 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf36eb-10e3-4d7b-9a98-06ca5dbd0932 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:23:01.652998 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf36eb-10e3-4d7b-9a98-06ca5dbd0932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:23:01.712502 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-8bcf36eb-10e3-4d7b-9a98-06ca5dbd0932 None None] MFA Rules not processed for user `61ba3008e5284162996252a821103d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:23:01.714131 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-8bcf36eb-10e3-4d7b-9a98-06ca5dbd0932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:23:01.774522 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:23:01.825570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1104/2200] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:23:01 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:23:01.842953 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366cbf2-534c-4bb9-993a-c36903e1863c service cinder] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:23:01.844569 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-d366cbf2-534c-4bb9-993a-c36903e1863c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:23:01.930321 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366cbf2-534c-4bb9-993a-c36903e1863c service cinder] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:23:01.938173 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec1541-7677-4686-956c-d071944c3026 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:23:01.938807 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec1541-7677-4686-956c-d071944c3026 service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:23:01.939120 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec1541-7677-4686-956c-d071944c3026 service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:23:01.939398 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec1541-7677-4686-956c-d071944c3026 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:23:01.941219 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c1ec1541-7677-4686-956c-d071944c3026 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:23:02.000312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:23:02.015554 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ec1541-7677-4686-956c-d071944c3026 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:23:02.017409 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ec1541-7677-4686-956c-d071944c3026 service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:23:02.027132 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1097/2201] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:23:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:23:11.822298 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=18 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:23:11.911447 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf36eb-10e3-4d7b-9a98-06ca5dbd0932 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:23:11.922327 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf36eb-10e3-4d7b-9a98-06ca5dbd0932 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:23:11.926477 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552593f-115f-4b11-8d67-73783cbacafe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:23:11.927140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552593f-115f-4b11-8d67-73783cbacafe service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:23:11.927429 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552593f-115f-4b11-8d67-73783cbacafe service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:23:11.927774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552593f-115f-4b11-8d67-73783cbacafe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:23:11.938696 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552593f-115f-4b11-8d67-73783cbacafe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:23:11.940200 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552593f-115f-4b11-8d67-73783cbacafe service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:23:11.950062 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1105/2202] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:23:11 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:23:12.022663 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=7 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:23:21.945616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=6 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:23:54.280105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86998add-4401-4a4d-81a3-39e370deed77 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:23:54.280636 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86998add-4401-4a4d-81a3-39e370deed77 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:23:54.280636 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86998add-4401-4a4d-81a3-39e370deed77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:23:54.300813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-86998add-4401-4a4d-81a3-39e370deed77 None None] MFA Rules not processed for user `a8fb7a9acefd4b418c992661037c2ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:23:54.302007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-86998add-4401-4a4d-81a3-39e370deed77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:23:54.356159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1098/2203] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 31 00:23:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:23:54.398571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552593f-115f-4b11-8d67-73783cbacafe service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:23:54.402283 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-9552593f-115f-4b11-8d67-73783cbacafe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:23:54.472093 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552593f-115f-4b11-8d67-73783cbacafe service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:23:54.477104 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8e062-55e0-4503-8003-fe9572c0aadc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd3cfbc8ebd4c5a83bef09efe10bc91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:23:54.477717 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8e062-55e0-4503-8003-fe9572c0aadc service swift] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:23:54.477993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8e062-55e0-4503-8003-fe9572c0aadc service swift] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:23:54.478257 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8e062-55e0-4503-8003-fe9572c0aadc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:23:54.487486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8e062-55e0-4503-8003-fe9572c0aadc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:23:54.489215 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8e062-55e0-4503-8003-fe9572c0aadc service swift] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:23:54.498155 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1106/2204] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 31 00:23:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:23:56.416592 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8ecc2-15c7-4632-aec3-f610ff3c9ee8 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:23:56.417357 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8ecc2-15c7-4632-aec3-f610ff3c9ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:23:56.417357 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8ecc2-15c7-4632-aec3-f610ff3c9ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:23:56.454175 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-add8ecc2-15c7-4632-aec3-f610ff3c9ee8 None None] MFA Rules not processed for user `f9add734db2a4adc9857040166c02675`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:23:56.455756 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-add8ecc2-15c7-4632-aec3-f610ff3c9ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:23:56.468258 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1099/2205] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:23:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:23:59.788557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599e724-fa11-4687-86b1-3f6bf2e7df9e None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:23:59.789189 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599e724-fa11-4687-86b1-3f6bf2e7df9e None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:23:59.789189 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599e724-fa11-4687-86b1-3f6bf2e7df9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:23:59.812112 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-b599e724-fa11-4687-86b1-3f6bf2e7df9e None None] MFA Rules not processed for user `3ff39cd0e766400f9d25e3ad465c4faf`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:23:59.813718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b599e724-fa11-4687-86b1-3f6bf2e7df9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:23:59.899113 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1107/2206] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:23:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:00.046765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add8ecc2-15c7-4632-aec3-f610ff3c9ee8 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:00.123765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add8ecc2-15c7-4632-aec3-f610ff3c9ee8 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:00.128951 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5746f-9b2a-401c-8b79-2d1285e0a41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:00.129716 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5746f-9b2a-401c-8b79-2d1285e0a41e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:00.130101 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5746f-9b2a-401c-8b79-2d1285e0a41e admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:00.130473 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5746f-9b2a-401c-8b79-2d1285e0a41e admin admin] PATH_INFO: `/v3/users/fa10b067020c43d2ae79aef38868f143` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:00.131762 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5746f-9b2a-401c-8b79-2d1285e0a41e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa10b067020c43d2ae79aef38868f143)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:00.134347 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5746f-9b2a-401c-8b79-2d1285e0a41e admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:00.206455 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-89a5746f-9b2a-401c-8b79-2d1285e0a41e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa10b067020c43d2ae79aef38868f143'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:00.221055 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-89a5746f-9b2a-401c-8b79-2d1285e0a41e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa10b067020c43d2ae79aef38868f143'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:00.229912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-89a5746f-9b2a-401c-8b79-2d1285e0a41e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa10b067020c43d2ae79aef38868f143'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:00.241438 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-89a5746f-9b2a-401c-8b79-2d1285e0a41e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa10b067020c43d2ae79aef38868f143'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:00.244337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-89a5746f-9b2a-401c-8b79-2d1285e0a41e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa10b067020c43d2ae79aef38868f143'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:00.252161 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1100/2207] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:24:00 2022] DELETE /identity/v3/users/fa10b067020c43d2ae79aef38868f143 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:00.256703 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65976c-25a2-4106-9d7c-c808420dd1ad None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:00.257439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65976c-25a2-4106-9d7c-c808420dd1ad None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:00.257867 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65976c-25a2-4106-9d7c-c808420dd1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:00.282183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-4b65976c-25a2-4106-9d7c-c808420dd1ad None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:00.284452 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4b65976c-25a2-4106-9d7c-c808420dd1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:00.359075 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1108/2208] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:24:00.369558 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5746f-9b2a-401c-8b79-2d1285e0a41e admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:00.372785 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-89a5746f-9b2a-401c-8b79-2d1285e0a41e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:00.397768 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:00.455899 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5746f-9b2a-401c-8b79-2d1285e0a41e admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:00.461334 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2af6f4-b8eb-4c87-95ae-ddee473c11e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:00.462074 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2af6f4-b8eb-4c87-95ae-ddee473c11e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:00.462422 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2af6f4-b8eb-4c87-95ae-ddee473c11e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:00.462779 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2af6f4-b8eb-4c87-95ae-ddee473c11e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:00.477012 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2af6f4-b8eb-4c87-95ae-ddee473c11e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:00.478732 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2af6f4-b8eb-4c87-95ae-ddee473c11e1 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:00.493474 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1101/2209] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:24:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:24:00.820698 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65976c-25a2-4106-9d7c-c808420dd1ad None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:00.822231 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4b65976c-25a2-4106-9d7c-c808420dd1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:00.840775 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65976c-25a2-4106-9d7c-c808420dd1ad None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:00.846408 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2dd560-7996-4b44-9716-83083b7e390a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:00.847320 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2dd560-7996-4b44-9716-83083b7e390a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:00.847774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2dd560-7996-4b44-9716-83083b7e390a admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:00.848046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2dd560-7996-4b44-9716-83083b7e390a admin admin] PATH_INFO: `/v3/projects/c6173289fa144f04b0121f79a70db124` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:00.849243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2dd560-7996-4b44-9716-83083b7e390a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6173289fa144f04b0121f79a70db124)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:00.851477 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2dd560-7996-4b44-9716-83083b7e390a admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:00.913895 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:00.945421 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aa2dd560-7996-4b44-9716-83083b7e390a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6173289fa144f04b0121f79a70db124', 'request_id': 'req-aa2dd560-7996-4b44-9716-83083b7e390a'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:00.952172 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aa2dd560-7996-4b44-9716-83083b7e390a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6173289fa144f04b0121f79a70db124', 'request_id': 'req-aa2dd560-7996-4b44-9716-83083b7e390a'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:00.975295 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aa2dd560-7996-4b44-9716-83083b7e390a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6173289fa144f04b0121f79a70db124', 'request_id': 'req-aa2dd560-7996-4b44-9716-83083b7e390a'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:01.015389 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aa2dd560-7996-4b44-9716-83083b7e390a admin admin] The token cache is being invalidate because project c6173289fa144f04b0121f79a70db124 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:24:01.015758 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aa2dd560-7996-4b44-9716-83083b7e390a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:01.016554 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aa2dd560-7996-4b44-9716-83083b7e390a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:01.019441 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1109/2210] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:24:00 2022] DELETE /identity/v3/projects/c6173289fa144f04b0121f79a70db124 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:01.023589 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2af6f4-b8eb-4c87-95ae-ddee473c11e1 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:01.052143 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2af6f4-b8eb-4c87-95ae-ddee473c11e1 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:01.124026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25a639-58ed-4ebe-8f5b-720da8486e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:01.124980 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25a639-58ed-4ebe-8f5b-720da8486e87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:01.125440 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25a639-58ed-4ebe-8f5b-720da8486e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:01.125717 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25a639-58ed-4ebe-8f5b-720da8486e87 admin admin] PATH_INFO: `/v3/users/7d45e94776cb4bccaff824f761c13239` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:01.127149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25a639-58ed-4ebe-8f5b-720da8486e87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d45e94776cb4bccaff824f761c13239)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:01.129587 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25a639-58ed-4ebe-8f5b-720da8486e87 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:01.223217 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ec25a639-58ed-4ebe-8f5b-720da8486e87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d45e94776cb4bccaff824f761c13239'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:01.265160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ec25a639-58ed-4ebe-8f5b-720da8486e87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d45e94776cb4bccaff824f761c13239'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:01.273982 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ec25a639-58ed-4ebe-8f5b-720da8486e87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d45e94776cb4bccaff824f761c13239'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:01.284806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ec25a639-58ed-4ebe-8f5b-720da8486e87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d45e94776cb4bccaff824f761c13239'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:01.285693 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ec25a639-58ed-4ebe-8f5b-720da8486e87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d45e94776cb4bccaff824f761c13239'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:01.294592 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1102/2211] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:24:01 2022] DELETE /identity/v3/users/7d45e94776cb4bccaff824f761c13239 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:01.719905 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2dd560-7996-4b44-9716-83083b7e390a admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:01.723312 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-aa2dd560-7996-4b44-9716-83083b7e390a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:01.742924 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2dd560-7996-4b44-9716-83083b7e390a admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:01.855293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c345565-ecb3-4958-9d92-3c0b9e191655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:01.856739 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c345565-ecb3-4958-9d92-3c0b9e191655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:01.857199 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c345565-ecb3-4958-9d92-3c0b9e191655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:01.857674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c345565-ecb3-4958-9d92-3c0b9e191655 admin admin] PATH_INFO: `/v3/projects/c32075d70bfb4c529e4a49c04f8c3fff` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:01.859237 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c345565-ecb3-4958-9d92-3c0b9e191655 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c32075d70bfb4c529e4a49c04f8c3fff)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:01.862165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c345565-ecb3-4958-9d92-3c0b9e191655 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:01.928031 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5c345565-ecb3-4958-9d92-3c0b9e191655 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c32075d70bfb4c529e4a49c04f8c3fff', 'request_id': 'req-5c345565-ecb3-4958-9d92-3c0b9e191655'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:01.937963 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5c345565-ecb3-4958-9d92-3c0b9e191655 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c32075d70bfb4c529e4a49c04f8c3fff', 'request_id': 'req-5c345565-ecb3-4958-9d92-3c0b9e191655'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:01.950076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5c345565-ecb3-4958-9d92-3c0b9e191655 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c32075d70bfb4c529e4a49c04f8c3fff', 'request_id': 'req-5c345565-ecb3-4958-9d92-3c0b9e191655'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:01.977160 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5c345565-ecb3-4958-9d92-3c0b9e191655 admin admin] The token cache is being invalidate because project c32075d70bfb4c529e4a49c04f8c3fff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:24:01.978266 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5c345565-ecb3-4958-9d92-3c0b9e191655 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:01.979206 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5c345565-ecb3-4958-9d92-3c0b9e191655 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:01.981388 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1110/2212] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:24:01 2022] DELETE /identity/v3/projects/c32075d70bfb4c529e4a49c04f8c3fff => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:01.987516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41480bf-5b4e-444e-8fe4-b77c3bcb7e7a None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:01.988164 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41480bf-5b4e-444e-8fe4-b77c3bcb7e7a None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:01.991225 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41480bf-5b4e-444e-8fe4-b77c3bcb7e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:02.020032 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-a41480bf-5b4e-444e-8fe4-b77c3bcb7e7a None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:02.021682 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-a41480bf-5b4e-444e-8fe4-b77c3bcb7e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:02.081058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:02.130099 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1103/2213] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:02.132762 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734c8d7-1b69-466a-a3c2-01ea6011af56 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:02.132762 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734c8d7-1b69-466a-a3c2-01ea6011af56 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:02.133748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734c8d7-1b69-466a-a3c2-01ea6011af56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:02.184559 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-f734c8d7-1b69-466a-a3c2-01ea6011af56 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:02.188018 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f734c8d7-1b69-466a-a3c2-01ea6011af56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:02.219493 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1111/2214] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:02.221328 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41480bf-5b4e-444e-8fe4-b77c3bcb7e7a None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:02.235926 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41480bf-5b4e-444e-8fe4-b77c3bcb7e7a None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:02.250970 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8f9f4-74ab-4823-ba10-4ed1b7174829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:02.251834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8f9f4-74ab-4823-ba10-4ed1b7174829 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:02.252189 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8f9f4-74ab-4823-ba10-4ed1b7174829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:02.252528 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8f9f4-74ab-4823-ba10-4ed1b7174829 admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:02.253250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e8f9f4-74ab-4823-ba10-4ed1b7174829 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:24:02.253642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e8f9f4-74ab-4823-ba10-4ed1b7174829 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:02.256021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e8f9f4-74ab-4823-ba10-4ed1b7174829 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:02.282831 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1104/2215] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:24:02.284689 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f734c8d7-1b69-466a-a3c2-01ea6011af56 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:02.294300 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8f9f4-74ab-4823-ba10-4ed1b7174829 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:02.297715 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f734c8d7-1b69-466a-a3c2-01ea6011af56 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:02.301143 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-e1e8f9f4-74ab-4823-ba10-4ed1b7174829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:02.316682 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b310f198-cd61-4050-88d5-5d40a4280860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:02.317399 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310f198-cd61-4050-88d5-5d40a4280860 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:02.317764 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310f198-cd61-4050-88d5-5d40a4280860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:02.318086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310f198-cd61-4050-88d5-5d40a4280860 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:02.318784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b310f198-cd61-4050-88d5-5d40a4280860 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:02.321004 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b310f198-cd61-4050-88d5-5d40a4280860 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:02.360534 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1112/2216] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:24:02 2022] POST /identity/v3/projects => generated 381 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:24:02.367146 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b310f198-cd61-4050-88d5-5d40a4280860 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:02.402451 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:02.441631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8f9f4-74ab-4823-ba10-4ed1b7174829 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:02.491088 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b310f198-cd61-4050-88d5-5d40a4280860 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:02.497612 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0d878-77c6-4fa3-8e0e-fd36e346d9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:02.498384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0d878-77c6-4fa3-8e0e-fd36e346d9a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:02.498647 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0d878-77c6-4fa3-8e0e-fd36e346d9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:02.498924 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0d878-77c6-4fa3-8e0e-fd36e346d9a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:02.499666 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0d878-77c6-4fa3-8e0e-fd36e346d9a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:02.502289 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0d878-77c6-4fa3-8e0e-fd36e346d9a7 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:02.534180 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1113/2217] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:24:02 2022] POST /identity/v3/users => generated 291 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:24:02.538014 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0d878-77c6-4fa3-8e0e-fd36e346d9a7 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:02.552894 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0d878-77c6-4fa3-8e0e-fd36e346d9a7 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:02.563804 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b1daa-a1e3-4848-81a1-180a97cce706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:02.566350 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b1daa-a1e3-4848-81a1-180a97cce706 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:02.567562 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b1daa-a1e3-4848-81a1-180a97cce706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:02.567846 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b1daa-a1e3-4848-81a1-180a97cce706 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:02.568754 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b1daa-a1e3-4848-81a1-180a97cce706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:02.570496 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b6338-a3c7-4faa-8e53-ddc6de0550f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdacdcb2451742938164fdd44cdbaebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:02.570813 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b1daa-a1e3-4848-81a1-180a97cce706 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:02.571167 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b6338-a3c7-4faa-8e53-ddc6de0550f9 service placement] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:02.571588 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b6338-a3c7-4faa-8e53-ddc6de0550f9 service placement] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:02.571870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b6338-a3c7-4faa-8e53-ddc6de0550f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:02.574118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-0e7b6338-a3c7-4faa-8e53-ddc6de0550f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:02.586668 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1114/2218] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:24:02.586668 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b1daa-a1e3-4848-81a1-180a97cce706 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:02.588365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4f8b1daa-a1e3-4848-81a1-180a97cce706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:02.590355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b6338-a3c7-4faa-8e53-ddc6de0550f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:02.592298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b6338-a3c7-4faa-8e53-ddc6de0550f9 service placement] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:02.608619 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b1daa-a1e3-4848-81a1-180a97cce706 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:02.611246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1105/2219] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:24:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:24:02.613050 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b6338-a3c7-4faa-8e53-ddc6de0550f9 service placement] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:02.615402 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e91bbe-cf03-43db-b18b-54d472114451 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdacdcb2451742938164fdd44cdbaebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:02.616053 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e91bbe-cf03-43db-b18b-54d472114451 service placement] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:02.616314 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e91bbe-cf03-43db-b18b-54d472114451 service placement] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:02.616604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e91bbe-cf03-43db-b18b-54d472114451 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:02.629579 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e91bbe-cf03-43db-b18b-54d472114451 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:02.629957 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b6338-a3c7-4faa-8e53-ddc6de0550f9 service placement] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:02.631235 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e91bbe-cf03-43db-b18b-54d472114451 service placement] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:02.636371 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a924ca-4e01-4fc8-9862-83788f497c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:02.636991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a924ca-4e01-4fc8-9862-83788f497c4f admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:02.637329 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a924ca-4e01-4fc8-9862-83788f497c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:02.637616 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a924ca-4e01-4fc8-9862-83788f497c4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:02.638333 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a924ca-4e01-4fc8-9862-83788f497c4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:02.640169 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a924ca-4e01-4fc8-9862-83788f497c4f admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:02.648068 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1115/2220] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:24:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:24:02.660228 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1106/2221] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:24:02.666995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e91bbe-cf03-43db-b18b-54d472114451 service placement] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:02.680657 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e91bbe-cf03-43db-b18b-54d472114451 service placement] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:02.686614 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45163dca-7871-4026-b7f8-4ca40dc86779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:02.687576 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45163dca-7871-4026-b7f8-4ca40dc86779 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:02.687975 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45163dca-7871-4026-b7f8-4ca40dc86779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:02.688422 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45163dca-7871-4026-b7f8-4ca40dc86779 admin admin] PATH_INFO: `/v3/projects/4edf913f47cb4bca9dcc9164c3bad95d/users/55bffc97172542bb9d02a568972b173f/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:02.717977 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45163dca-7871-4026-b7f8-4ca40dc86779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4edf913f47cb4bca9dcc9164c3bad95d, user_id=55bffc97172542bb9d02a568972b173f, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:02.722058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45163dca-7871-4026-b7f8-4ca40dc86779 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:02.747615 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1116/2222] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:24:02 2022] PUT /identity/v3/projects/4edf913f47cb4bca9dcc9164c3bad95d/users/55bffc97172542bb9d02a568972b173f/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:02.758184 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7450813-8317-43b4-bcc7-d80c4c0b487c None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:02.758474 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7450813-8317-43b4-bcc7-d80c4c0b487c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:02.758753 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7450813-8317-43b4-bcc7-d80c4c0b487c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:02.806560 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-b7450813-8317-43b4-bcc7-d80c4c0b487c None None] MFA Rules not processed for user `55bffc97172542bb9d02a568972b173f`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:02.808157 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b7450813-8317-43b4-bcc7-d80c4c0b487c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:02.858752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:02.909803 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1107/2223] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:02.914019 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45163dca-7871-4026-b7f8-4ca40dc86779 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:03.027182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45163dca-7871-4026-b7f8-4ca40dc86779 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:03.040388 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2793095-8960-41c6-b795-ba50ec64f0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:03.041091 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2793095-8960-41c6-b795-ba50ec64f0a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:03.041445 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2793095-8960-41c6-b795-ba50ec64f0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:03.041818 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2793095-8960-41c6-b795-ba50ec64f0a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:03.042506 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2793095-8960-41c6-b795-ba50ec64f0a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:03.044578 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2793095-8960-41c6-b795-ba50ec64f0a2 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:03.089368 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1117/2224] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:24:02 2022] POST /identity/v3/projects => generated 381 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:24:03.093040 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7450813-8317-43b4-bcc7-d80c4c0b487c None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:03.105443 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7450813-8317-43b4-bcc7-d80c4c0b487c None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:03.214351 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586beff9-0919-4199-afd1-194f432dea66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:03.217043 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586beff9-0919-4199-afd1-194f432dea66 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:03.217487 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586beff9-0919-4199-afd1-194f432dea66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:03.217775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586beff9-0919-4199-afd1-194f432dea66 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:03.218427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586beff9-0919-4199-afd1-194f432dea66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:03.220626 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586beff9-0919-4199-afd1-194f432dea66 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:03.262513 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1108/2225] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:24:03 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:24:03.266512 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2793095-8960-41c6-b795-ba50ec64f0a2 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:03.291431 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2793095-8960-41c6-b795-ba50ec64f0a2 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:03.299657 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b782a7-a1ee-4f2d-9a66-c68ac6ef1daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:03.300287 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b782a7-a1ee-4f2d-9a66-c68ac6ef1daa admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:03.300571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b782a7-a1ee-4f2d-9a66-c68ac6ef1daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:03.300843 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b782a7-a1ee-4f2d-9a66-c68ac6ef1daa admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:03.301496 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b782a7-a1ee-4f2d-9a66-c68ac6ef1daa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:03.303318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b782a7-a1ee-4f2d-9a66-c68ac6ef1daa admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:03.319940 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1118/2226] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:24:03.321889 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586beff9-0919-4199-afd1-194f432dea66 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:03.348345 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586beff9-0919-4199-afd1-194f432dea66 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:03.366244 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac5056-3f8e-46dd-b7d6-325e821d4678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:03.367090 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac5056-3f8e-46dd-b7d6-325e821d4678 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:03.367776 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac5056-3f8e-46dd-b7d6-325e821d4678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:03.368707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac5056-3f8e-46dd-b7d6-325e821d4678 admin admin] PATH_INFO: `/v3/projects/588c930e902a4028a081beeec7c9dee1/users/7e7e02637bd747f6a25d3247fa67cec1/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:03.393181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac5056-3f8e-46dd-b7d6-325e821d4678 admin admin] RBAC: Authorizing `identity:create_grant(project_id=588c930e902a4028a081beeec7c9dee1, user_id=7e7e02637bd747f6a25d3247fa67cec1, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:03.396316 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac5056-3f8e-46dd-b7d6-325e821d4678 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:03.420403 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1109/2227] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:24:03 2022] PUT /identity/v3/projects/588c930e902a4028a081beeec7c9dee1/users/7e7e02637bd747f6a25d3247fa67cec1/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:03.424364 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b782a7-a1ee-4f2d-9a66-c68ac6ef1daa admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:03.437421 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b782a7-a1ee-4f2d-9a66-c68ac6ef1daa admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:03.476557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:03.509440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658af14e-237f-41de-889f-0676095c2eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:03.509440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658af14e-237f-41de-889f-0676095c2eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:03.509440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658af14e-237f-41de-889f-0676095c2eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:03.509440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658af14e-237f-41de-889f-0676095c2eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:03.510249 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658af14e-237f-41de-889f-0676095c2eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:03.512028 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658af14e-237f-41de-889f-0676095c2eb8 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:03.530290 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1119/2228] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:24:03.531775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac5056-3f8e-46dd-b7d6-325e821d4678 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:03.544933 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac5056-3f8e-46dd-b7d6-325e821d4678 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:03.550687 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2e772-6bdc-4279-badb-8005b3457609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:03.551418 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2e772-6bdc-4279-badb-8005b3457609 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:03.551722 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2e772-6bdc-4279-badb-8005b3457609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:03.552030 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2e772-6bdc-4279-badb-8005b3457609 admin admin] PATH_INFO: `/v3/projects/588c930e902a4028a081beeec7c9dee1/users/7e7e02637bd747f6a25d3247fa67cec1/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:03.554182 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b2e772-6bdc-4279-badb-8005b3457609 admin admin] RBAC: Authorizing `identity:create_grant(project_id=588c930e902a4028a081beeec7c9dee1, user_id=7e7e02637bd747f6a25d3247fa67cec1, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:03.557125 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b2e772-6bdc-4279-badb-8005b3457609 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:03.573513 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1110/2229] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:24:03 2022] PUT /identity/v3/projects/588c930e902a4028a081beeec7c9dee1/users/7e7e02637bd747f6a25d3247fa67cec1/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:03.585643 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3c9f2-c6ad-4f13-8164-7e7c09225eaf None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:03.585981 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3c9f2-c6ad-4f13-8164-7e7c09225eaf None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:03.586255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3c9f2-c6ad-4f13-8164-7e7c09225eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:03.646535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-3cb3c9f2-c6ad-4f13-8164-7e7c09225eaf None None] MFA Rules not processed for user `7e7e02637bd747f6a25d3247fa67cec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:03.647876 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-3cb3c9f2-c6ad-4f13-8164-7e7c09225eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:03.767255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1120/2230] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:03 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:03.786254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2e772-6bdc-4279-badb-8005b3457609 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:03.788315 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-11b2e772-6bdc-4279-badb-8005b3457609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:03.858827 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2e772-6bdc-4279-badb-8005b3457609 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:03.863973 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea2291-f235-4ed8-a313-7fc2a0206813 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:03.864662 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea2291-f235-4ed8-a313-7fc2a0206813 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:03.864936 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea2291-f235-4ed8-a313-7fc2a0206813 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:03.865683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea2291-f235-4ed8-a313-7fc2a0206813 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:03.878462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea2291-f235-4ed8-a313-7fc2a0206813 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:03.880156 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea2291-f235-4ed8-a313-7fc2a0206813 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:03.890670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1111/2231] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:24:03 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:24:04.536634 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3c9f2-c6ad-4f13-8164-7e7c09225eaf None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:04.562194 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3c9f2-c6ad-4f13-8164-7e7c09225eaf None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:04.576521 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158c59b-5637-4eae-a698-b6e474cf50ab tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e7e02637bd747f6a25d3247fa67cec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '588c930e902a4028a081beeec7c9dee1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:04.577466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158c59b-5637-4eae-a698-b6e474cf50ab tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:04.577923 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158c59b-5637-4eae-a698-b6e474cf50ab tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:04.578354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158c59b-5637-4eae-a698-b6e474cf50ab tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:04.579213 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158c59b-5637-4eae-a698-b6e474cf50ab tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:24:04.579721 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158c59b-5637-4eae-a698-b6e474cf50ab tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:04.581885 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158c59b-5637-4eae-a698-b6e474cf50ab tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:04.597958 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1121/2232] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:24:04.602100 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea2291-f235-4ed8-a313-7fc2a0206813 service cinder] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:04.614693 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea2291-f235-4ed8-a313-7fc2a0206813 service cinder] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:04.636805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c6335d-8d83-4a67-94b6-c0568628ea25 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e7e02637bd747f6a25d3247fa67cec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '588c930e902a4028a081beeec7c9dee1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:04.637534 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6335d-8d83-4a67-94b6-c0568628ea25 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:04.637887 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6335d-8d83-4a67-94b6-c0568628ea25 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:04.638233 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6335d-8d83-4a67-94b6-c0568628ea25 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:04.639513 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c6335d-8d83-4a67-94b6-c0568628ea25 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:04.642053 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c6335d-8d83-4a67-94b6-c0568628ea25 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:04.702905 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1112/2233] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:24:04 2022] POST /identity/v3/projects => generated 355 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:24:04.707546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158c59b-5637-4eae-a698-b6e474cf50ab tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:04.725950 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158c59b-5637-4eae-a698-b6e474cf50ab tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:04.855579 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb643e-23be-4608-8781-59922a9ae805 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e7e02637bd747f6a25d3247fa67cec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '588c930e902a4028a081beeec7c9dee1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:04.856216 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb643e-23be-4608-8781-59922a9ae805 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:04.856621 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb643e-23be-4608-8781-59922a9ae805 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:04.857127 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb643e-23be-4608-8781-59922a9ae805 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:04.857859 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb643e-23be-4608-8781-59922a9ae805 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:24:04.858263 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb643e-23be-4608-8781-59922a9ae805 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:04.861035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb643e-23be-4608-8781-59922a9ae805 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:04.875407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1122/2234] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:24:04.918617 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c6335d-8d83-4a67-94b6-c0568628ea25 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:04.937021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c6335d-8d83-4a67-94b6-c0568628ea25 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:04.945550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f2031-8071-4972-949c-db395c399599 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e7e02637bd747f6a25d3247fa67cec1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '588c930e902a4028a081beeec7c9dee1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:04.946474 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f2031-8071-4972-949c-db395c399599 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:04.946953 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f2031-8071-4972-949c-db395c399599 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:04.947477 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f2031-8071-4972-949c-db395c399599 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] PATH_INFO: `/v3/projects/44731d7d6ac648aa9d445bf46b9bf5df` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:04.948820 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679f2031-8071-4972-949c-db395c399599 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] RBAC: Authorizing `identity:delete_project(project_id=44731d7d6ac648aa9d445bf46b9bf5df)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:04.951107 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679f2031-8071-4972-949c-db395c399599 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:04.966707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:05.068245 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-679f2031-8071-4972-949c-db395c399599 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44731d7d6ac648aa9d445bf46b9bf5df', 'request_id': 'req-679f2031-8071-4972-949c-db395c399599'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:05.080720 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-679f2031-8071-4972-949c-db395c399599 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44731d7d6ac648aa9d445bf46b9bf5df', 'request_id': 'req-679f2031-8071-4972-949c-db395c399599'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:05.091947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-679f2031-8071-4972-949c-db395c399599 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44731d7d6ac648aa9d445bf46b9bf5df', 'request_id': 'req-679f2031-8071-4972-949c-db395c399599'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:05.123093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-679f2031-8071-4972-949c-db395c399599 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] The token cache is being invalidate because project 44731d7d6ac648aa9d445bf46b9bf5df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:24:05.123579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-679f2031-8071-4972-949c-db395c399599 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:05.133717 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-679f2031-8071-4972-949c-db395c399599 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:05.137211 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1113/2235] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:24:04 2022] DELETE /identity/v3/projects/44731d7d6ac648aa9d445bf46b9bf5df => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:05.582230 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb643e-23be-4608-8781-59922a9ae805 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:05.584774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-c8cb643e-23be-4608-8781-59922a9ae805 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:05.649577 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:05.708099 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb643e-23be-4608-8781-59922a9ae805 tempest-VolumeQuotaClassesTest-1613394634 tempest-VolumeQuotaClassesTest-1613394634-project-admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:05.886937 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef3eae-92a5-4d38-8ae2-a8ff17462790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:05.887924 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef3eae-92a5-4d38-8ae2-a8ff17462790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:05.888528 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef3eae-92a5-4d38-8ae2-a8ff17462790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:05.888966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef3eae-92a5-4d38-8ae2-a8ff17462790 admin admin] PATH_INFO: `/v3/users/55bffc97172542bb9d02a568972b173f` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:05.890338 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ef3eae-92a5-4d38-8ae2-a8ff17462790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55bffc97172542bb9d02a568972b173f)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:05.893262 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ef3eae-92a5-4d38-8ae2-a8ff17462790 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:06.020321 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-b4ef3eae-92a5-4d38-8ae2-a8ff17462790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55bffc97172542bb9d02a568972b173f'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:06.041583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-b4ef3eae-92a5-4d38-8ae2-a8ff17462790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55bffc97172542bb9d02a568972b173f'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:06.053624 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-b4ef3eae-92a5-4d38-8ae2-a8ff17462790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55bffc97172542bb9d02a568972b173f'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:06.056519 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-b4ef3eae-92a5-4d38-8ae2-a8ff17462790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55bffc97172542bb9d02a568972b173f'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:06.057855 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-b4ef3eae-92a5-4d38-8ae2-a8ff17462790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55bffc97172542bb9d02a568972b173f'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:06.097937 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1123/2236] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:24:05 2022] DELETE /identity/v3/users/55bffc97172542bb9d02a568972b173f => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:06.102526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cbd2b0-94ed-4317-9f14-b13acca04fb6 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:06.103523 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cbd2b0-94ed-4317-9f14-b13acca04fb6 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:06.103809 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cbd2b0-94ed-4317-9f14-b13acca04fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:06.160469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-b1cbd2b0-94ed-4317-9f14-b13acca04fb6 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:06.176137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b1cbd2b0-94ed-4317-9f14-b13acca04fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:06.284402 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1114/2237] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:24:06.299856 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef3eae-92a5-4d38-8ae2-a8ff17462790 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:06.301520 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b4ef3eae-92a5-4d38-8ae2-a8ff17462790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:06.400515 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef3eae-92a5-4d38-8ae2-a8ff17462790 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:06.406163 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3e41d-0d63-45c2-9d94-0c1b75f84580 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:06.406935 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3e41d-0d63-45c2-9d94-0c1b75f84580 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:06.407343 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3e41d-0d63-45c2-9d94-0c1b75f84580 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:06.407736 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3e41d-0d63-45c2-9d94-0c1b75f84580 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:06.429884 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d3e41d-0d63-45c2-9d94-0c1b75f84580 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:06.431543 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d3e41d-0d63-45c2-9d94-0c1b75f84580 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:06.452743 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1124/2238] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:24:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:24:07.268454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cbd2b0-94ed-4317-9f14-b13acca04fb6 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:07.271787 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b1cbd2b0-94ed-4317-9f14-b13acca04fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:07.330250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cbd2b0-94ed-4317-9f14-b13acca04fb6 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:07.370793 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b30f3c-6955-44a9-926f-e759efa6927a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:07.373099 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b30f3c-6955-44a9-926f-e759efa6927a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:07.373099 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b30f3c-6955-44a9-926f-e759efa6927a admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:07.373099 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b30f3c-6955-44a9-926f-e759efa6927a admin admin] PATH_INFO: `/v3/projects/4edf913f47cb4bca9dcc9164c3bad95d` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:07.374372 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b30f3c-6955-44a9-926f-e759efa6927a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4edf913f47cb4bca9dcc9164c3bad95d)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:07.376414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b30f3c-6955-44a9-926f-e759efa6927a admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:07.495779 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-56b30f3c-6955-44a9-926f-e759efa6927a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4edf913f47cb4bca9dcc9164c3bad95d', 'request_id': 'req-56b30f3c-6955-44a9-926f-e759efa6927a'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:07.510079 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-56b30f3c-6955-44a9-926f-e759efa6927a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4edf913f47cb4bca9dcc9164c3bad95d', 'request_id': 'req-56b30f3c-6955-44a9-926f-e759efa6927a'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:07.518366 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-56b30f3c-6955-44a9-926f-e759efa6927a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4edf913f47cb4bca9dcc9164c3bad95d', 'request_id': 'req-56b30f3c-6955-44a9-926f-e759efa6927a'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:07.534148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-56b30f3c-6955-44a9-926f-e759efa6927a admin admin] The token cache is being invalidate because project 4edf913f47cb4bca9dcc9164c3bad95d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:24:07.534764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-56b30f3c-6955-44a9-926f-e759efa6927a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:07.535955 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-56b30f3c-6955-44a9-926f-e759efa6927a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:07.538884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1115/2239] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:24:07 2022] DELETE /identity/v3/projects/4edf913f47cb4bca9dcc9164c3bad95d => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:07.542429 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3e41d-0d63-45c2-9d94-0c1b75f84580 service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:07.561270 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3e41d-0d63-45c2-9d94-0c1b75f84580 service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:07.568007 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:07.681055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c05f9c-f994-4684-a793-41d7786986bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:07.681901 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c05f9c-f994-4684-a793-41d7786986bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:07.682281 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c05f9c-f994-4684-a793-41d7786986bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:07.683318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c05f9c-f994-4684-a793-41d7786986bc admin admin] PATH_INFO: `/v3/users/7e7e02637bd747f6a25d3247fa67cec1` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:07.684752 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c05f9c-f994-4684-a793-41d7786986bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e7e02637bd747f6a25d3247fa67cec1)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:07.687106 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c05f9c-f994-4684-a793-41d7786986bc admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:07.792065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-21c05f9c-f994-4684-a793-41d7786986bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e7e02637bd747f6a25d3247fa67cec1'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:07.802802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-21c05f9c-f994-4684-a793-41d7786986bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e7e02637bd747f6a25d3247fa67cec1'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:07.819969 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-21c05f9c-f994-4684-a793-41d7786986bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e7e02637bd747f6a25d3247fa67cec1'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:07.821585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-21c05f9c-f994-4684-a793-41d7786986bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e7e02637bd747f6a25d3247fa67cec1'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:07.822415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-21c05f9c-f994-4684-a793-41d7786986bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e7e02637bd747f6a25d3247fa67cec1'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:07.858404 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1125/2240] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:24:07 2022] DELETE /identity/v3/users/7e7e02637bd747f6a25d3247fa67cec1 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:08.222448 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b30f3c-6955-44a9-926f-e759efa6927a admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:08.224854 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-56b30f3c-6955-44a9-926f-e759efa6927a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:08.240252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b30f3c-6955-44a9-926f-e759efa6927a admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:08.262443 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:08.363334 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283deba-0ffa-44da-bcea-21f5dc46359a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:08.364220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283deba-0ffa-44da-bcea-21f5dc46359a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:08.364220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283deba-0ffa-44da-bcea-21f5dc46359a admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:08.365438 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283deba-0ffa-44da-bcea-21f5dc46359a admin admin] PATH_INFO: `/v3/projects/588c930e902a4028a081beeec7c9dee1` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:08.369359 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1283deba-0ffa-44da-bcea-21f5dc46359a admin admin] RBAC: Authorizing `identity:delete_project(project_id=588c930e902a4028a081beeec7c9dee1)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:08.370828 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1283deba-0ffa-44da-bcea-21f5dc46359a admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:08.477964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-1283deba-0ffa-44da-bcea-21f5dc46359a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '588c930e902a4028a081beeec7c9dee1', 'request_id': 'req-1283deba-0ffa-44da-bcea-21f5dc46359a'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:08.491465 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-1283deba-0ffa-44da-bcea-21f5dc46359a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '588c930e902a4028a081beeec7c9dee1', 'request_id': 'req-1283deba-0ffa-44da-bcea-21f5dc46359a'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:08.498312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-1283deba-0ffa-44da-bcea-21f5dc46359a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '588c930e902a4028a081beeec7c9dee1', 'request_id': 'req-1283deba-0ffa-44da-bcea-21f5dc46359a'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:08.511589 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-1283deba-0ffa-44da-bcea-21f5dc46359a admin admin] The token cache is being invalidate because project 588c930e902a4028a081beeec7c9dee1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:24:08.511981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-1283deba-0ffa-44da-bcea-21f5dc46359a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:08.512578 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-1283deba-0ffa-44da-bcea-21f5dc46359a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:08.514134 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1116/2241] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:24:08 2022] DELETE /identity/v3/projects/588c930e902a4028a081beeec7c9dee1 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:08.519638 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9d199-5e18-4cc7-9e75-d23e3f7f9dc3 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:08.519929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9d199-5e18-4cc7-9e75-d23e3f7f9dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:08.521491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9d199-5e18-4cc7-9e75-d23e3f7f9dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:08.542140 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-f8b9d199-5e18-4cc7-9e75-d23e3f7f9dc3 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:08.543584 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f8b9d199-5e18-4cc7-9e75-d23e3f7f9dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:08.612699 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1126/2242] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:08.622086 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a38ff-0cb1-4d1e-ae4e-9b1ea1fe6d77 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:08.622435 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a38ff-0cb1-4d1e-ae4e-9b1ea1fe6d77 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:08.622774 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a38ff-0cb1-4d1e-ae4e-9b1ea1fe6d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:08.646402 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-764a38ff-0cb1-4d1e-ae4e-9b1ea1fe6d77 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:08.647685 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-764a38ff-0cb1-4d1e-ae4e-9b1ea1fe6d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:08.662638 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1117/2243] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:08.664292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9d199-5e18-4cc7-9e75-d23e3f7f9dc3 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:08.675633 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9d199-5e18-4cc7-9e75-d23e3f7f9dc3 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:08.682313 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924aafbb-c887-4135-8b8e-262faa0ad88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:08.684630 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924aafbb-c887-4135-8b8e-262faa0ad88f admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:08.685012 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924aafbb-c887-4135-8b8e-262faa0ad88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:08.685530 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924aafbb-c887-4135-8b8e-262faa0ad88f admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:08.686442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924aafbb-c887-4135-8b8e-262faa0ad88f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:24:08.686942 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924aafbb-c887-4135-8b8e-262faa0ad88f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:08.693683 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924aafbb-c887-4135-8b8e-262faa0ad88f admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:08.714785 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1127/2244] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:24:08.716529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a38ff-0cb1-4d1e-ae4e-9b1ea1fe6d77 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:08.732673 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a38ff-0cb1-4d1e-ae4e-9b1ea1fe6d77 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:08.739623 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed285d-3831-4424-b705-9feb1ef7bc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:08.740368 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed285d-3831-4424-b705-9feb1ef7bc23 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:08.741012 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed285d-3831-4424-b705-9feb1ef7bc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:08.741347 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed285d-3831-4424-b705-9feb1ef7bc23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:08.742080 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed285d-3831-4424-b705-9feb1ef7bc23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:08.744121 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed285d-3831-4424-b705-9feb1ef7bc23 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:08.780148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1118/2245] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:24:08 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:24:08.785769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924aafbb-c887-4135-8b8e-262faa0ad88f admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:08.797858 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924aafbb-c887-4135-8b8e-262faa0ad88f admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:08.812639 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:08.876459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63286b21-1bca-4198-b7d8-8eb410e3ed1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:08.878733 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63286b21-1bca-4198-b7d8-8eb410e3ed1e admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:08.879160 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63286b21-1bca-4198-b7d8-8eb410e3ed1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:08.879538 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63286b21-1bca-4198-b7d8-8eb410e3ed1e admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:08.880306 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63286b21-1bca-4198-b7d8-8eb410e3ed1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:08.882533 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63286b21-1bca-4198-b7d8-8eb410e3ed1e admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:08.928658 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1128/2246] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:24:08 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:24:08.930064 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed285d-3831-4424-b705-9feb1ef7bc23 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:08.947100 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed285d-3831-4424-b705-9feb1ef7bc23 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:08.954492 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a37ba3-e750-4d41-b1ce-ac1d2eb10c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:08.955929 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a37ba3-e750-4d41-b1ce-ac1d2eb10c69 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:08.956319 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a37ba3-e750-4d41-b1ce-ac1d2eb10c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:08.957088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a37ba3-e750-4d41-b1ce-ac1d2eb10c69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:08.958086 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a37ba3-e750-4d41-b1ce-ac1d2eb10c69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:08.959802 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a37ba3-e750-4d41-b1ce-ac1d2eb10c69 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:08.975083 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1119/2247] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:24:08.977253 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63286b21-1bca-4198-b7d8-8eb410e3ed1e admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:08.988038 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63286b21-1bca-4198-b7d8-8eb410e3ed1e admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:08.999591 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc3445-d32e-414f-8aac-41b27ac0ba19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:09.000456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc3445-d32e-414f-8aac-41b27ac0ba19 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:09.000832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc3445-d32e-414f-8aac-41b27ac0ba19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:09.001245 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc3445-d32e-414f-8aac-41b27ac0ba19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:09.002685 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc3445-d32e-414f-8aac-41b27ac0ba19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:09.005067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc3445-d32e-414f-8aac-41b27ac0ba19 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:09.021941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1129/2248] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:24:09.026731 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a37ba3-e750-4d41-b1ce-ac1d2eb10c69 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:09.038115 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a37ba3-e750-4d41-b1ce-ac1d2eb10c69 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:09.045350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68baf1-297b-40f3-9094-4ad66d8c2d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:09.046019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68baf1-297b-40f3-9094-4ad66d8c2d14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:09.046299 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68baf1-297b-40f3-9094-4ad66d8c2d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:09.046579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68baf1-297b-40f3-9094-4ad66d8c2d14 admin admin] PATH_INFO: `/v3/projects/4d52091fcebd42d4b7b25d2c515216e6/users/ce5d00df950c42f88863b9ed358a37df/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:09.063938 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be68baf1-297b-40f3-9094-4ad66d8c2d14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d52091fcebd42d4b7b25d2c515216e6, user_id=ce5d00df950c42f88863b9ed358a37df, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:09.067273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be68baf1-297b-40f3-9094-4ad66d8c2d14 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:09.088602 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1120/2249] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:24:09 2022] PUT /identity/v3/projects/4d52091fcebd42d4b7b25d2c515216e6/users/ce5d00df950c42f88863b9ed358a37df/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:09.094529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa17295-f6e2-46a9-8d13-0e6cec40f50c None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:09.094913 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa17295-f6e2-46a9-8d13-0e6cec40f50c None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:09.095327 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa17295-f6e2-46a9-8d13-0e6cec40f50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:09.134712 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-0fa17295-f6e2-46a9-8d13-0e6cec40f50c None None] MFA Rules not processed for user `ce5d00df950c42f88863b9ed358a37df`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:09.136134 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-0fa17295-f6e2-46a9-8d13-0e6cec40f50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:09.223082 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1130/2250] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:09.226411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68baf1-297b-40f3-9094-4ad66d8c2d14 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:09.280226 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:09.308951 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68baf1-297b-40f3-9094-4ad66d8c2d14 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:09.315133 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a6af3-71c6-4d1c-a8ec-425554779293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:09.316526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a6af3-71c6-4d1c-a8ec-425554779293 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:09.316827 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a6af3-71c6-4d1c-a8ec-425554779293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:09.317100 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a6af3-71c6-4d1c-a8ec-425554779293 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:09.317837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a6af3-71c6-4d1c-a8ec-425554779293 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:09.320028 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a6af3-71c6-4d1c-a8ec-425554779293 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:09.356557 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1121/2251] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:24:09 2022] POST /identity/v3/projects => generated 387 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:24:09.361114 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa17295-f6e2-46a9-8d13-0e6cec40f50c None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:09.387418 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa17295-f6e2-46a9-8d13-0e6cec40f50c None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:09.508512 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac3d8c-2527-481b-8f20-7795283a8237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:09.508512 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac3d8c-2527-481b-8f20-7795283a8237 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:09.508512 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac3d8c-2527-481b-8f20-7795283a8237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:09.509053 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac3d8c-2527-481b-8f20-7795283a8237 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:09.509671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac3d8c-2527-481b-8f20-7795283a8237 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:09.511804 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac3d8c-2527-481b-8f20-7795283a8237 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:09.571752 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1131/2252] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:24:09 2022] POST /identity/v3/users => generated 294 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:24:09.576393 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a6af3-71c6-4d1c-a8ec-425554779293 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:09.590290 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a6af3-71c6-4d1c-a8ec-425554779293 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:09.596287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3757a7e-c0c8-459c-b0c1-914d811722c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:09.597061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3757a7e-c0c8-459c-b0c1-914d811722c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:09.597740 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3757a7e-c0c8-459c-b0c1-914d811722c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:09.598178 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3757a7e-c0c8-459c-b0c1-914d811722c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:09.598905 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3757a7e-c0c8-459c-b0c1-914d811722c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:09.601285 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3757a7e-c0c8-459c-b0c1-914d811722c0 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:09.626861 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1122/2253] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:24:09.631684 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac3d8c-2527-481b-8f20-7795283a8237 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:09.644195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac3d8c-2527-481b-8f20-7795283a8237 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:09.688200 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b83b3-b0f9-42e3-a0a8-d83836cc3caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:09.688890 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b83b3-b0f9-42e3-a0a8-d83836cc3caa admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:09.689195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b83b3-b0f9-42e3-a0a8-d83836cc3caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:09.689616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b83b3-b0f9-42e3-a0a8-d83836cc3caa admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:09.697929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303b83b3-b0f9-42e3-a0a8-d83836cc3caa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:09.699678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303b83b3-b0f9-42e3-a0a8-d83836cc3caa admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:09.722044 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:09.747917 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1132/2254] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:24:09.752115 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3757a7e-c0c8-459c-b0c1-914d811722c0 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:09.771332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3757a7e-c0c8-459c-b0c1-914d811722c0 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:09.777702 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b798b-4ccc-4e8c-b6e3-2b9c27c02e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:09.778785 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b798b-4ccc-4e8c-b6e3-2b9c27c02e1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:09.779365 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b798b-4ccc-4e8c-b6e3-2b9c27c02e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:09.780102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b798b-4ccc-4e8c-b6e3-2b9c27c02e1c admin admin] PATH_INFO: `/v3/projects/9baefcb37def4f5396bb35c257cf3833/users/0f39c29adf244e65b6a1e12fec48034b/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:09.803107 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b798b-4ccc-4e8c-b6e3-2b9c27c02e1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9baefcb37def4f5396bb35c257cf3833, user_id=0f39c29adf244e65b6a1e12fec48034b, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:09.806362 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b798b-4ccc-4e8c-b6e3-2b9c27c02e1c admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:09.831664 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1123/2255] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:24:09 2022] PUT /identity/v3/projects/9baefcb37def4f5396bb35c257cf3833/users/0f39c29adf244e65b6a1e12fec48034b/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:24:09.842594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b71e7-6e20-4b50-b1f0-5d36c0560e2b None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:09.842939 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b71e7-6e20-4b50-b1f0-5d36c0560e2b None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:09.843237 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b71e7-6e20-4b50-b1f0-5d36c0560e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:09.899623 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-210b71e7-6e20-4b50-b1f0-5d36c0560e2b None None] MFA Rules not processed for user `0f39c29adf244e65b6a1e12fec48034b`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:09.901367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-210b71e7-6e20-4b50-b1f0-5d36c0560e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:10.044769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1133/2256] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:10.051954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b798b-4ccc-4e8c-b6e3-2b9c27c02e1c admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:10.179135 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b798b-4ccc-4e8c-b6e3-2b9c27c02e1c admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:10.188018 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f107f-a33e-437d-908a-140a1e94d625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:10.189209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f107f-a33e-437d-908a-140a1e94d625 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:10.189819 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f107f-a33e-437d-908a-140a1e94d625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:10.190319 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f107f-a33e-437d-908a-140a1e94d625 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:10.191237 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f107f-a33e-437d-908a-140a1e94d625 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:10.193998 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f107f-a33e-437d-908a-140a1e94d625 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:10.282822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1124/2257] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:24:10 2022] POST /identity/v3/projects => generated 387 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:24:10.286869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b71e7-6e20-4b50-b1f0-5d36c0560e2b None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:10.310751 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b71e7-6e20-4b50-b1f0-5d36c0560e2b None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:10.427550 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51244df-dd10-4a8b-9d64-ba3f1566284c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:10.428698 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51244df-dd10-4a8b-9d64-ba3f1566284c admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:10.429146 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51244df-dd10-4a8b-9d64-ba3f1566284c admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:10.429601 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51244df-dd10-4a8b-9d64-ba3f1566284c admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:10.430244 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51244df-dd10-4a8b-9d64-ba3f1566284c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:10.432221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51244df-dd10-4a8b-9d64-ba3f1566284c admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:10.457203 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1134/2258] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:24:10 2022] POST /identity/v3/users => generated 300 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:24:10.461014 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f107f-a33e-437d-908a-140a1e94d625 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:10.473152 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f107f-a33e-437d-908a-140a1e94d625 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:10.479003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167bb10f-aebf-48aa-bcb6-bcf724b18eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:10.480137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bb10f-aebf-48aa-bcb6-bcf724b18eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:10.480497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bb10f-aebf-48aa-bcb6-bcf724b18eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:10.481213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bb10f-aebf-48aa-bcb6-bcf724b18eb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:10.482238 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167bb10f-aebf-48aa-bcb6-bcf724b18eb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:10.484547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167bb10f-aebf-48aa-bcb6-bcf724b18eb1 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:10.498805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1125/2259] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:24:10.503501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51244df-dd10-4a8b-9d64-ba3f1566284c admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:10.514190 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51244df-dd10-4a8b-9d64-ba3f1566284c admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:10.521248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb74833-c04b-4431-8375-109fefbeafd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:10.522007 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb74833-c04b-4431-8375-109fefbeafd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:10.522355 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb74833-c04b-4431-8375-109fefbeafd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:10.522693 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb74833-c04b-4431-8375-109fefbeafd0 admin admin] PATH_INFO: `/v3/projects/19e13662feb043b7b6beee3e4f9eef03/users/ab82ec7cea6b42b4bf7ff0fc84519d64/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:10.543997 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb74833-c04b-4431-8375-109fefbeafd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19e13662feb043b7b6beee3e4f9eef03, user_id=ab82ec7cea6b42b4bf7ff0fc84519d64, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:10.546869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb74833-c04b-4431-8375-109fefbeafd0 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:10.561691 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1135/2260] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:24:10 2022] PUT /identity/v3/projects/19e13662feb043b7b6beee3e4f9eef03/users/ab82ec7cea6b42b4bf7ff0fc84519d64/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:10.566455 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167bb10f-aebf-48aa-bcb6-bcf724b18eb1 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:10.577969 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167bb10f-aebf-48aa-bcb6-bcf724b18eb1 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:10.640583 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac709a-11d7-4349-97b9-1e506cacb210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:10.641241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac709a-11d7-4349-97b9-1e506cacb210 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:10.641590 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac709a-11d7-4349-97b9-1e506cacb210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:10.641931 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac709a-11d7-4349-97b9-1e506cacb210 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:10.643618 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac709a-11d7-4349-97b9-1e506cacb210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:10.645490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac709a-11d7-4349-97b9-1e506cacb210 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:10.660815 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1126/2261] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:24:10.662123 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb74833-c04b-4431-8375-109fefbeafd0 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:10.675096 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb74833-c04b-4431-8375-109fefbeafd0 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:10.681963 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172dcdae-222e-48b6-95fe-d51f0a7c273f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:10.682731 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dcdae-222e-48b6-95fe-d51f0a7c273f admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:10.683092 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dcdae-222e-48b6-95fe-d51f0a7c273f admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:10.683443 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dcdae-222e-48b6-95fe-d51f0a7c273f admin admin] PATH_INFO: `/v3/projects/19e13662feb043b7b6beee3e4f9eef03/users/ab82ec7cea6b42b4bf7ff0fc84519d64/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:10.685491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172dcdae-222e-48b6-95fe-d51f0a7c273f admin admin] RBAC: Authorizing `identity:create_grant(project_id=19e13662feb043b7b6beee3e4f9eef03, user_id=ab82ec7cea6b42b4bf7ff0fc84519d64, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:10.688286 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172dcdae-222e-48b6-95fe-d51f0a7c273f admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:10.701055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1136/2262] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:24:10 2022] PUT /identity/v3/projects/19e13662feb043b7b6beee3e4f9eef03/users/ab82ec7cea6b42b4bf7ff0fc84519d64/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:10.711810 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e43491-667a-49da-b3db-0662939893a0 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:10.712183 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e43491-667a-49da-b3db-0662939893a0 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:10.712542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e43491-667a-49da-b3db-0662939893a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:10.721861 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:10.748246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-71e43491-667a-49da-b3db-0662939893a0 None None] MFA Rules not processed for user `ab82ec7cea6b42b4bf7ff0fc84519d64`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:10.749561 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-71e43491-667a-49da-b3db-0662939893a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:10.840650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1127/2263] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:10 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:10.854174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172dcdae-222e-48b6-95fe-d51f0a7c273f admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:10.856228 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-172dcdae-222e-48b6-95fe-d51f0a7c273f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:10.917469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:10.934692 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172dcdae-222e-48b6-95fe-d51f0a7c273f admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:10.938834 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c413a-779b-42e1-bc4b-1ab070e5a929 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:10.939552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c413a-779b-42e1-bc4b-1ab070e5a929 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:10.939839 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c413a-779b-42e1-bc4b-1ab070e5a929 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:10.940113 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c413a-779b-42e1-bc4b-1ab070e5a929 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:10.951428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c413a-779b-42e1-bc4b-1ab070e5a929 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:10.953021 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c413a-779b-42e1-bc4b-1ab070e5a929 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:10.965200 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1137/2264] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:24:10 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:24:11.763624 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e43491-667a-49da-b3db-0662939893a0 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:11.765085 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-71e43491-667a-49da-b3db-0662939893a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:11.778515 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e43491-667a-49da-b3db-0662939893a0 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:11.784066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69978b8-e94f-4809-8fdf-8c40875fe961 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:11.784977 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69978b8-e94f-4809-8fdf-8c40875fe961 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:11.785263 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69978b8-e94f-4809-8fdf-8c40875fe961 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:11.785534 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69978b8-e94f-4809-8fdf-8c40875fe961 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:11.854843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69978b8-e94f-4809-8fdf-8c40875fe961 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:11.857984 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69978b8-e94f-4809-8fdf-8c40875fe961 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:11.870234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1128/2265] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:24:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:24:15.282033 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0057a98-7ff8-4114-8898-1998bef39cf7 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:15.282539 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0057a98-7ff8-4114-8898-1998bef39cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:15.282632 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0057a98-7ff8-4114-8898-1998bef39cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:15.302782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-b0057a98-7ff8-4114-8898-1998bef39cf7 None None] MFA Rules not processed for user `a8fb7a9acefd4b418c992661037c2ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:15.304022 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b0057a98-7ff8-4114-8898-1998bef39cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:15.367964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1138/2266] 10.222.0.67 () {58 vars in 967 bytes} [Wed Aug 31 00:24:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:15.381583 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69978b8-e94f-4809-8fdf-8c40875fe961 service cinder] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:15.383010 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b69978b8-e94f-4809-8fdf-8c40875fe961 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:15.469385 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69978b8-e94f-4809-8fdf-8c40875fe961 service cinder] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:15.475211 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27fea3-c050-4340-b13a-971aff790e14 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd3cfbc8ebd4c5a83bef09efe10bc91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:15.475824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27fea3-c050-4340-b13a-971aff790e14 service swift] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:15.476105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27fea3-c050-4340-b13a-971aff790e14 service swift] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:15.476380 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27fea3-c050-4340-b13a-971aff790e14 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:15.485629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27fea3-c050-4340-b13a-971aff790e14 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:15.487487 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27fea3-c050-4340-b13a-971aff790e14 service swift] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:15.496105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1129/2267] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 31 00:24:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:24:18.207583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0057a98-7ff8-4114-8898-1998bef39cf7 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:18.217396 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0057a98-7ff8-4114-8898-1998bef39cf7 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:18.221257 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407b9c4-9768-4d20-864d-ced725ead3e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:18.221873 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407b9c4-9768-4d20-864d-ced725ead3e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:18.222156 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407b9c4-9768-4d20-864d-ced725ead3e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:18.222427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407b9c4-9768-4d20-864d-ced725ead3e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:18.288372 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407b9c4-9768-4d20-864d-ced725ead3e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:18.289823 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407b9c4-9768-4d20-864d-ced725ead3e1 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:18.298903 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1139/2268] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:24:18 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:24:18.577263 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27fea3-c050-4340-b13a-971aff790e14 service swift] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:18.578841 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-2d27fea3-c050-4340-b13a-971aff790e14 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:18.604438 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:18.653334 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27fea3-c050-4340-b13a-971aff790e14 service swift] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:18.658267 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb2308-7e9c-426c-84c1-97a45181cbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:18.658900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb2308-7e9c-426c-84c1-97a45181cbe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:18.659210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb2308-7e9c-426c-84c1-97a45181cbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:18.659621 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb2308-7e9c-426c-84c1-97a45181cbe1 admin admin] PATH_INFO: `/v3/users/95e5a1b739a54c3aa4dcb53e5bbe0452` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:18.660763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bb2308-7e9c-426c-84c1-97a45181cbe1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95e5a1b739a54c3aa4dcb53e5bbe0452)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:18.662673 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bb2308-7e9c-426c-84c1-97a45181cbe1 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:18.715755 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-c4bb2308-7e9c-426c-84c1-97a45181cbe1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95e5a1b739a54c3aa4dcb53e5bbe0452'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:18.733860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-c4bb2308-7e9c-426c-84c1-97a45181cbe1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95e5a1b739a54c3aa4dcb53e5bbe0452'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:18.756517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-c4bb2308-7e9c-426c-84c1-97a45181cbe1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95e5a1b739a54c3aa4dcb53e5bbe0452'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:18.769031 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-c4bb2308-7e9c-426c-84c1-97a45181cbe1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95e5a1b739a54c3aa4dcb53e5bbe0452'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:18.769770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-c4bb2308-7e9c-426c-84c1-97a45181cbe1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95e5a1b739a54c3aa4dcb53e5bbe0452'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:18.777525 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1130/2269] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:24:18 2022] DELETE /identity/v3/users/95e5a1b739a54c3aa4dcb53e5bbe0452 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:18.985311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407b9c4-9768-4d20-864d-ced725ead3e1 service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:19.007707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407b9c4-9768-4d20-864d-ced725ead3e1 service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:19.081685 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed05dde-4844-4102-b0ba-30f269d3afe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:19.083181 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed05dde-4844-4102-b0ba-30f269d3afe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:19.083478 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed05dde-4844-4102-b0ba-30f269d3afe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:19.083736 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed05dde-4844-4102-b0ba-30f269d3afe5 admin admin] PATH_INFO: `/v3/projects/d94c6a3ef756488eac765456d78ff335` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:19.085040 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed05dde-4844-4102-b0ba-30f269d3afe5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d94c6a3ef756488eac765456d78ff335)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:19.089068 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed05dde-4844-4102-b0ba-30f269d3afe5 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:19.130048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:19.153127 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aed05dde-4844-4102-b0ba-30f269d3afe5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd94c6a3ef756488eac765456d78ff335', 'request_id': 'req-aed05dde-4844-4102-b0ba-30f269d3afe5'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:19.160980 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aed05dde-4844-4102-b0ba-30f269d3afe5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd94c6a3ef756488eac765456d78ff335', 'request_id': 'req-aed05dde-4844-4102-b0ba-30f269d3afe5'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:19.167903 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aed05dde-4844-4102-b0ba-30f269d3afe5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd94c6a3ef756488eac765456d78ff335', 'request_id': 'req-aed05dde-4844-4102-b0ba-30f269d3afe5'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:19.186628 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aed05dde-4844-4102-b0ba-30f269d3afe5 admin admin] The token cache is being invalidate because project d94c6a3ef756488eac765456d78ff335 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:24:19.186923 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aed05dde-4844-4102-b0ba-30f269d3afe5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:19.187576 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aed05dde-4844-4102-b0ba-30f269d3afe5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:19.189135 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1140/2270] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:24:18 2022] DELETE /identity/v3/projects/d94c6a3ef756488eac765456d78ff335 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:19.194231 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61eb6d9-7437-48d8-b507-8e3bbaa87c9e None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:19.194518 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61eb6d9-7437-48d8-b507-8e3bbaa87c9e None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:19.194795 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61eb6d9-7437-48d8-b507-8e3bbaa87c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:19.215408 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-f61eb6d9-7437-48d8-b507-8e3bbaa87c9e None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:19.216630 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f61eb6d9-7437-48d8-b507-8e3bbaa87c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:19.293049 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1131/2271] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:19.303129 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0c748-5942-476f-be27-e251e226c5cd None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:19.303654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0c748-5942-476f-be27-e251e226c5cd None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:19.303885 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0c748-5942-476f-be27-e251e226c5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:19.326874 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-f3a0c748-5942-476f-be27-e251e226c5cd None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:19.328118 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f3a0c748-5942-476f-be27-e251e226c5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:19.336842 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1141/2272] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:24:19.341237 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61eb6d9-7437-48d8-b507-8e3bbaa87c9e None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:19.351728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61eb6d9-7437-48d8-b507-8e3bbaa87c9e None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:19.356804 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e04778-46bb-4acf-843f-d0d5c6142606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:19.357461 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e04778-46bb-4acf-843f-d0d5c6142606 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:19.357744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e04778-46bb-4acf-843f-d0d5c6142606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:19.358020 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e04778-46bb-4acf-843f-d0d5c6142606 admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:19.358671 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e04778-46bb-4acf-843f-d0d5c6142606 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:24:19.358977 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e04778-46bb-4acf-843f-d0d5c6142606 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:19.360605 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e04778-46bb-4acf-843f-d0d5c6142606 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:19.373548 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1132/2273] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:24:19.379383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0c748-5942-476f-be27-e251e226c5cd None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:19.389849 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0c748-5942-476f-be27-e251e226c5cd None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:19.395356 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78528d57-8d03-4294-a9bd-ea7ec277d258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:19.395947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78528d57-8d03-4294-a9bd-ea7ec277d258 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:19.396223 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78528d57-8d03-4294-a9bd-ea7ec277d258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:19.396514 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78528d57-8d03-4294-a9bd-ea7ec277d258 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:19.397118 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78528d57-8d03-4294-a9bd-ea7ec277d258 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:19.399005 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78528d57-8d03-4294-a9bd-ea7ec277d258 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:19.430880 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1142/2274] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:24:19 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:24:19.435179 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e04778-46bb-4acf-843f-d0d5c6142606 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:19.446379 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e04778-46bb-4acf-843f-d0d5c6142606 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:19.467066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:19.506631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9635a-1cd6-4635-9a57-65ac93ebcc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:19.507265 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9635a-1cd6-4635-9a57-65ac93ebcc49 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:19.507544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9635a-1cd6-4635-9a57-65ac93ebcc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:19.507817 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9635a-1cd6-4635-9a57-65ac93ebcc49 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:19.508459 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc9635a-1cd6-4635-9a57-65ac93ebcc49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:19.510394 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc9635a-1cd6-4635-9a57-65ac93ebcc49 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:19.532135 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1133/2275] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:24:19 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:24:19.536804 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78528d57-8d03-4294-a9bd-ea7ec277d258 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:19.547237 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78528d57-8d03-4294-a9bd-ea7ec277d258 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:19.552931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26bade1-cd10-4569-9506-01aa85be7d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:19.553535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26bade1-cd10-4569-9506-01aa85be7d01 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:19.553817 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26bade1-cd10-4569-9506-01aa85be7d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:19.554091 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26bade1-cd10-4569-9506-01aa85be7d01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:19.554733 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26bade1-cd10-4569-9506-01aa85be7d01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:19.556379 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26bade1-cd10-4569-9506-01aa85be7d01 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:19.567441 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1143/2276] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:24:19.571489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9635a-1cd6-4635-9a57-65ac93ebcc49 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:19.581489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9635a-1cd6-4635-9a57-65ac93ebcc49 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:19.586178 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb26c8-3e0c-4bd7-a74a-8a00f0542633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:19.586768 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb26c8-3e0c-4bd7-a74a-8a00f0542633 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:19.587122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb26c8-3e0c-4bd7-a74a-8a00f0542633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:19.587341 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb26c8-3e0c-4bd7-a74a-8a00f0542633 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:19.587976 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb26c8-3e0c-4bd7-a74a-8a00f0542633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:19.589575 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb26c8-3e0c-4bd7-a74a-8a00f0542633 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:19.599994 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1134/2277] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:24:19.604973 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26bade1-cd10-4569-9506-01aa85be7d01 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:19.615094 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26bade1-cd10-4569-9506-01aa85be7d01 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:19.619831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ffd4f-d887-48db-8e13-c1e6e3bba6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:19.620478 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ffd4f-d887-48db-8e13-c1e6e3bba6e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:19.620761 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ffd4f-d887-48db-8e13-c1e6e3bba6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:19.621038 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ffd4f-d887-48db-8e13-c1e6e3bba6e4 admin admin] PATH_INFO: `/v3/projects/e32cbe1a05ce414fb55f1de8bf71581c/users/b03932102b7541f89a60a9c535a42e76/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:19.635997 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6ffd4f-d887-48db-8e13-c1e6e3bba6e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e32cbe1a05ce414fb55f1de8bf71581c, user_id=b03932102b7541f89a60a9c535a42e76, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:19.638667 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6ffd4f-d887-48db-8e13-c1e6e3bba6e4 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:19.651986 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1144/2278] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:24:19 2022] PUT /identity/v3/projects/e32cbe1a05ce414fb55f1de8bf71581c/users/b03932102b7541f89a60a9c535a42e76/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:24:19.656954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21844af9-1238-4daf-b2e3-13e78bf83624 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:19.657249 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21844af9-1238-4daf-b2e3-13e78bf83624 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:19.657530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21844af9-1238-4daf-b2e3-13e78bf83624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:19.678908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-21844af9-1238-4daf-b2e3-13e78bf83624 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:19.680152 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-21844af9-1238-4daf-b2e3-13e78bf83624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:19.739426 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1135/2279] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:19.750343 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ffd4f-d887-48db-8e13-c1e6e3bba6e4 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:19.752244 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-8e6ffd4f-d887-48db-8e13-c1e6e3bba6e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:19.809597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ffd4f-d887-48db-8e13-c1e6e3bba6e4 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:19.813225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef124e-f40b-41ac-b0f0-c896bde1e14b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:19.813831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef124e-f40b-41ac-b0f0-c896bde1e14b service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:19.814110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef124e-f40b-41ac-b0f0-c896bde1e14b service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:19.814382 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef124e-f40b-41ac-b0f0-c896bde1e14b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:19.823417 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ef124e-f40b-41ac-b0f0-c896bde1e14b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:19.824846 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ef124e-f40b-41ac-b0f0-c896bde1e14b service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:19.834211 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1145/2280] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:24:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:24:20.551318 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb791b-0410-4856-9b79-b412a44ded76 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:20.552123 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb791b-0410-4856-9b79-b412a44ded76 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:20.552123 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb791b-0410-4856-9b79-b412a44ded76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:20.592455 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-6ffb791b-0410-4856-9b79-b412a44ded76 None None] MFA Rules not processed for user `b03932102b7541f89a60a9c535a42e76`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:20.593790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-6ffb791b-0410-4856-9b79-b412a44ded76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:20.682923 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1136/2281] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:20.695994 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef124e-f40b-41ac-b0f0-c896bde1e14b service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:20.700492 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-c3ef124e-f40b-41ac-b0f0-c896bde1e14b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:20.774171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef124e-f40b-41ac-b0f0-c896bde1e14b service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:20.783126 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba5d91-3d23-41b1-a8ae-3c076d8910ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:20.784081 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba5d91-3d23-41b1-a8ae-3c076d8910ca service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:20.784677 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba5d91-3d23-41b1-a8ae-3c076d8910ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:20.784954 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba5d91-3d23-41b1-a8ae-3c076d8910ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:20.802752 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daba5d91-3d23-41b1-a8ae-3c076d8910ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:20.804253 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daba5d91-3d23-41b1-a8ae-3c076d8910ca service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:20.811330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:20.820641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1146/2282] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:24:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:24:20.838528 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb791b-0410-4856-9b79-b412a44ded76 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:20.840509 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-6ffb791b-0410-4856-9b79-b412a44ded76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:20.857582 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb791b-0410-4856-9b79-b412a44ded76 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:20.863094 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad23c0-1bed-4fed-b832-7b0a82a9536a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:20.863567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad23c0-1bed-4fed-b832-7b0a82a9536a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:20.863842 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad23c0-1bed-4fed-b832-7b0a82a9536a admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:20.864174 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad23c0-1bed-4fed-b832-7b0a82a9536a admin admin] PATH_INFO: `/v3/users/ce5d00df950c42f88863b9ed358a37df` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:20.865802 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad23c0-1bed-4fed-b832-7b0a82a9536a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce5d00df950c42f88863b9ed358a37df)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:20.868295 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad23c0-1bed-4fed-b832-7b0a82a9536a admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:20.905295 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:20.934098 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-c3ad23c0-1bed-4fed-b832-7b0a82a9536a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce5d00df950c42f88863b9ed358a37df'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:20.959978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-c3ad23c0-1bed-4fed-b832-7b0a82a9536a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce5d00df950c42f88863b9ed358a37df'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:20.978204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-c3ad23c0-1bed-4fed-b832-7b0a82a9536a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce5d00df950c42f88863b9ed358a37df'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:20.993636 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-c3ad23c0-1bed-4fed-b832-7b0a82a9536a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce5d00df950c42f88863b9ed358a37df'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:20.996896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-c3ad23c0-1bed-4fed-b832-7b0a82a9536a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce5d00df950c42f88863b9ed358a37df'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:21.018822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1137/2283] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:24:20 2022] DELETE /identity/v3/users/ce5d00df950c42f88863b9ed358a37df => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:21.020656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6c687-20e8-49f8-b1a8-bb83fb3d7b4c None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:21.021015 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6c687-20e8-49f8-b1a8-bb83fb3d7b4c None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:21.021321 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6c687-20e8-49f8-b1a8-bb83fb3d7b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:21.045971 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-38e6c687-20e8-49f8-b1a8-bb83fb3d7b4c None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:21.047492 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-38e6c687-20e8-49f8-b1a8-bb83fb3d7b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:21.124819 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1147/2284] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:21.132543 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad23c0-1bed-4fed-b832-7b0a82a9536a admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:21.134035 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c3ad23c0-1bed-4fed-b832-7b0a82a9536a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:21.250226 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad23c0-1bed-4fed-b832-7b0a82a9536a admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:21.254585 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d7acc-219a-4518-a6e3-ee59c732276b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:21.255261 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d7acc-219a-4518-a6e3-ee59c732276b service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:21.255541 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d7acc-219a-4518-a6e3-ee59c732276b service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:21.255810 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d7acc-219a-4518-a6e3-ee59c732276b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:21.267517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39d7acc-219a-4518-a6e3-ee59c732276b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:21.269835 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39d7acc-219a-4518-a6e3-ee59c732276b service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:21.283120 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1138/2285] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:24:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:24:21.567062 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6c687-20e8-49f8-b1a8-bb83fb3d7b4c None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:21.568471 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-38e6c687-20e8-49f8-b1a8-bb83fb3d7b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:21.580864 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6c687-20e8-49f8-b1a8-bb83fb3d7b4c None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:21.585909 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aaebd0-f02f-45c5-bc04-f4746da9a597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:21.586549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aaebd0-f02f-45c5-bc04-f4746da9a597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:21.586831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aaebd0-f02f-45c5-bc04-f4746da9a597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:21.587138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aaebd0-f02f-45c5-bc04-f4746da9a597 admin admin] PATH_INFO: `/v3/projects/4d52091fcebd42d4b7b25d2c515216e6` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:21.588278 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aaebd0-f02f-45c5-bc04-f4746da9a597 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d52091fcebd42d4b7b25d2c515216e6)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:21.590187 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aaebd0-f02f-45c5-bc04-f4746da9a597 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:21.642321 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-55aaebd0-f02f-45c5-bc04-f4746da9a597 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d52091fcebd42d4b7b25d2c515216e6', 'request_id': 'req-55aaebd0-f02f-45c5-bc04-f4746da9a597'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:21.649785 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-55aaebd0-f02f-45c5-bc04-f4746da9a597 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d52091fcebd42d4b7b25d2c515216e6', 'request_id': 'req-55aaebd0-f02f-45c5-bc04-f4746da9a597'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:21.657673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-55aaebd0-f02f-45c5-bc04-f4746da9a597 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d52091fcebd42d4b7b25d2c515216e6', 'request_id': 'req-55aaebd0-f02f-45c5-bc04-f4746da9a597'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:21.672373 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-55aaebd0-f02f-45c5-bc04-f4746da9a597 admin admin] The token cache is being invalidate because project 4d52091fcebd42d4b7b25d2c515216e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:24:21.672900 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-55aaebd0-f02f-45c5-bc04-f4746da9a597 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:21.673554 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-55aaebd0-f02f-45c5-bc04-f4746da9a597 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:21.676180 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1148/2286] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:24:21 2022] DELETE /identity/v3/projects/4d52091fcebd42d4b7b25d2c515216e6 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:21.679978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d7acc-219a-4518-a6e3-ee59c732276b service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:21.692304 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d7acc-219a-4518-a6e3-ee59c732276b service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:21.780254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e65fc-529a-447a-9930-56d0e45528fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:21.781224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e65fc-529a-447a-9930-56d0e45528fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:21.781503 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e65fc-529a-447a-9930-56d0e45528fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:21.781791 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e65fc-529a-447a-9930-56d0e45528fd admin admin] PATH_INFO: `/v3/users/0f39c29adf244e65b6a1e12fec48034b` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:21.782920 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e65fc-529a-447a-9930-56d0e45528fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f39c29adf244e65b6a1e12fec48034b)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:21.784984 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e65fc-529a-447a-9930-56d0e45528fd admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:21.841988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9d6e65fc-529a-447a-9930-56d0e45528fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f39c29adf244e65b6a1e12fec48034b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:21.843893 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:21.856785 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9d6e65fc-529a-447a-9930-56d0e45528fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f39c29adf244e65b6a1e12fec48034b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:21.864688 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9d6e65fc-529a-447a-9930-56d0e45528fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f39c29adf244e65b6a1e12fec48034b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:21.873575 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9d6e65fc-529a-447a-9930-56d0e45528fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f39c29adf244e65b6a1e12fec48034b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:21.874225 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-9d6e65fc-529a-447a-9930-56d0e45528fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f39c29adf244e65b6a1e12fec48034b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:21.881403 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1139/2287] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:24:21 2022] DELETE /identity/v3/users/0f39c29adf244e65b6a1e12fec48034b => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:22.161493 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aaebd0-f02f-45c5-bc04-f4746da9a597 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:22.163246 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-55aaebd0-f02f-45c5-bc04-f4746da9a597 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:22.178275 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aaebd0-f02f-45c5-bc04-f4746da9a597 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:22.237840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa756894-11e6-4b18-b16e-2c0a931a0c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:22.238389 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa756894-11e6-4b18-b16e-2c0a931a0c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:22.238670 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa756894-11e6-4b18-b16e-2c0a931a0c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:22.238944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa756894-11e6-4b18-b16e-2c0a931a0c38 admin admin] PATH_INFO: `/v3/projects/9baefcb37def4f5396bb35c257cf3833` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:22.240121 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa756894-11e6-4b18-b16e-2c0a931a0c38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9baefcb37def4f5396bb35c257cf3833)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:22.242082 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa756894-11e6-4b18-b16e-2c0a931a0c38 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:22.303100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:22.325433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aa756894-11e6-4b18-b16e-2c0a931a0c38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9baefcb37def4f5396bb35c257cf3833', 'request_id': 'req-aa756894-11e6-4b18-b16e-2c0a931a0c38'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:22.331381 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aa756894-11e6-4b18-b16e-2c0a931a0c38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9baefcb37def4f5396bb35c257cf3833', 'request_id': 'req-aa756894-11e6-4b18-b16e-2c0a931a0c38'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:22.338798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aa756894-11e6-4b18-b16e-2c0a931a0c38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9baefcb37def4f5396bb35c257cf3833', 'request_id': 'req-aa756894-11e6-4b18-b16e-2c0a931a0c38'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:22.354315 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aa756894-11e6-4b18-b16e-2c0a931a0c38 admin admin] The token cache is being invalidate because project 9baefcb37def4f5396bb35c257cf3833 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:24:22.354821 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aa756894-11e6-4b18-b16e-2c0a931a0c38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:22.355745 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aa756894-11e6-4b18-b16e-2c0a931a0c38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:22.357276 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1149/2288] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:24:22 2022] DELETE /identity/v3/projects/9baefcb37def4f5396bb35c257cf3833 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:22.361466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e65fc-529a-447a-9930-56d0e45528fd admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:22.363725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-9d6e65fc-529a-447a-9930-56d0e45528fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:22.377154 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e65fc-529a-447a-9930-56d0e45528fd admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:22.438301 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680d4e5-cce9-4e75-aba5-62b0d78ca523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:22.438940 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680d4e5-cce9-4e75-aba5-62b0d78ca523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:22.439244 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680d4e5-cce9-4e75-aba5-62b0d78ca523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:22.439514 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680d4e5-cce9-4e75-aba5-62b0d78ca523 admin admin] PATH_INFO: `/v3/users/ab82ec7cea6b42b4bf7ff0fc84519d64` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:22.441606 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680d4e5-cce9-4e75-aba5-62b0d78ca523 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab82ec7cea6b42b4bf7ff0fc84519d64)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:22.443567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680d4e5-cce9-4e75-aba5-62b0d78ca523 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:22.493713 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-2680d4e5-cce9-4e75-aba5-62b0d78ca523 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab82ec7cea6b42b4bf7ff0fc84519d64'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:22.508431 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-2680d4e5-cce9-4e75-aba5-62b0d78ca523 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab82ec7cea6b42b4bf7ff0fc84519d64'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:22.515814 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-2680d4e5-cce9-4e75-aba5-62b0d78ca523 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab82ec7cea6b42b4bf7ff0fc84519d64'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:22.524979 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-2680d4e5-cce9-4e75-aba5-62b0d78ca523 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab82ec7cea6b42b4bf7ff0fc84519d64'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:22.525708 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-2680d4e5-cce9-4e75-aba5-62b0d78ca523 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab82ec7cea6b42b4bf7ff0fc84519d64'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:22.532705 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1140/2289] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:24:22 2022] DELETE /identity/v3/users/ab82ec7cea6b42b4bf7ff0fc84519d64 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:22.837018 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa756894-11e6-4b18-b16e-2c0a931a0c38 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:22.839664 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-aa756894-11e6-4b18-b16e-2c0a931a0c38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:22.866574 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa756894-11e6-4b18-b16e-2c0a931a0c38 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:22.942528 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaabc598-f353-491b-8f29-6f5628b995f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:22.943592 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabc598-f353-491b-8f29-6f5628b995f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:22.944023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabc598-f353-491b-8f29-6f5628b995f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:22.944467 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabc598-f353-491b-8f29-6f5628b995f4 admin admin] PATH_INFO: `/v3/projects/19e13662feb043b7b6beee3e4f9eef03` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:22.945938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaabc598-f353-491b-8f29-6f5628b995f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19e13662feb043b7b6beee3e4f9eef03)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:22.948931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaabc598-f353-491b-8f29-6f5628b995f4 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:23.015109 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aaabc598-f353-491b-8f29-6f5628b995f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19e13662feb043b7b6beee3e4f9eef03', 'request_id': 'req-aaabc598-f353-491b-8f29-6f5628b995f4'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:23.022534 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aaabc598-f353-491b-8f29-6f5628b995f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19e13662feb043b7b6beee3e4f9eef03', 'request_id': 'req-aaabc598-f353-491b-8f29-6f5628b995f4'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:23.030435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aaabc598-f353-491b-8f29-6f5628b995f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19e13662feb043b7b6beee3e4f9eef03', 'request_id': 'req-aaabc598-f353-491b-8f29-6f5628b995f4'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:23.051719 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aaabc598-f353-491b-8f29-6f5628b995f4 admin admin] The token cache is being invalidate because project 19e13662feb043b7b6beee3e4f9eef03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:24:23.052059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aaabc598-f353-491b-8f29-6f5628b995f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:23.062807 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-aaabc598-f353-491b-8f29-6f5628b995f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:23.067113 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1150/2290] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:24:22 2022] DELETE /identity/v3/projects/19e13662feb043b7b6beee3e4f9eef03 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:23.073263 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f19227-fa92-4941-b487-ac9cd938d6a5 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:23.073772 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f19227-fa92-4941-b487-ac9cd938d6a5 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:23.073911 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f19227-fa92-4941-b487-ac9cd938d6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:23.094109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-f1f19227-fa92-4941-b487-ac9cd938d6a5 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:23.095388 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f1f19227-fa92-4941-b487-ac9cd938d6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:23.144754 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:23.171200 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1141/2291] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:23.178024 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93db146-2416-498f-8c7d-3879d84f97c9 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:23.178550 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93db146-2416-498f-8c7d-3879d84f97c9 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:23.178651 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93db146-2416-498f-8c7d-3879d84f97c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:23.199676 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-a93db146-2416-498f-8c7d-3879d84f97c9 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:23.200898 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a93db146-2416-498f-8c7d-3879d84f97c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:23.209571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1151/2292] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:23.214132 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f19227-fa92-4941-b487-ac9cd938d6a5 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:23.224688 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f19227-fa92-4941-b487-ac9cd938d6a5 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:23.229338 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ef9d7-aa34-4c8e-97ec-349259d57091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:23.230601 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ef9d7-aa34-4c8e-97ec-349259d57091 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:23.230925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ef9d7-aa34-4c8e-97ec-349259d57091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:23.231288 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ef9d7-aa34-4c8e-97ec-349259d57091 admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:23.231947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ef9d7-aa34-4c8e-97ec-349259d57091 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:24:23.232250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ef9d7-aa34-4c8e-97ec-349259d57091 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:23.233867 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ef9d7-aa34-4c8e-97ec-349259d57091 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:23.246311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1142/2293] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:24:23.250645 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93db146-2416-498f-8c7d-3879d84f97c9 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:23.261434 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93db146-2416-498f-8c7d-3879d84f97c9 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:23.266590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6d59b-00ea-453d-9877-7198163a9a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:23.267177 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6d59b-00ea-453d-9877-7198163a9a14 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:23.267459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6d59b-00ea-453d-9877-7198163a9a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:23.267726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6d59b-00ea-453d-9877-7198163a9a14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:23.268433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6d59b-00ea-453d-9877-7198163a9a14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:23.270616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6d59b-00ea-453d-9877-7198163a9a14 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:23.301184 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1152/2294] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:24:23 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:24:23.304811 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ef9d7-aa34-4c8e-97ec-349259d57091 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:23.316640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ef9d7-aa34-4c8e-97ec-349259d57091 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:23.370361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a3ef2-2fda-47a4-9b89-8513fba24f3e None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:23.370650 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a3ef2-2fda-47a4-9b89-8513fba24f3e None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:23.370929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a3ef2-2fda-47a4-9b89-8513fba24f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:23.379061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38393af0-4afb-432e-b24c-e8b4b1890d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:23.379800 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38393af0-4afb-432e-b24c-e8b4b1890d85 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:23.380086 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38393af0-4afb-432e-b24c-e8b4b1890d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:23.380357 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38393af0-4afb-432e-b24c-e8b4b1890d85 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:23.380991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38393af0-4afb-432e-b24c-e8b4b1890d85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:23.382904 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38393af0-4afb-432e-b24c-e8b4b1890d85 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:23.390829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-f72a3ef2-2fda-47a4-9b89-8513fba24f3e None None] MFA Rules not processed for user `b03932102b7541f89a60a9c535a42e76`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:23.394407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f72a3ef2-2fda-47a4-9b89-8513fba24f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:23.406007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1143/2295] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:24:23 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:24:23.406202 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:23.410480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38393af0-4afb-432e-b24c-e8b4b1890d85 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:23.422495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38393af0-4afb-432e-b24c-e8b4b1890d85 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:23.430270 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2b4bb-be0e-44c3-8cb6-bb094db56f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:23.430989 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2b4bb-be0e-44c3-8cb6-bb094db56f83 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:23.431517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2b4bb-be0e-44c3-8cb6-bb094db56f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:23.431993 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2b4bb-be0e-44c3-8cb6-bb094db56f83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:23.432722 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b2b4bb-be0e-44c3-8cb6-bb094db56f83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:23.434869 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b2b4bb-be0e-44c3-8cb6-bb094db56f83 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:23.450941 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1144/2296] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:24:23.451942 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2b4bb-be0e-44c3-8cb6-bb094db56f83 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:23.464122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2b4bb-be0e-44c3-8cb6-bb094db56f83 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:23.466909 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1153/2297] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:24:23.470082 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113c46e-4371-4ae7-b920-251834e07877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:23.472012 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113c46e-4371-4ae7-b920-251834e07877 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:23.472479 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113c46e-4371-4ae7-b920-251834e07877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:23.472912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113c46e-4371-4ae7-b920-251834e07877 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:23.473839 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2113c46e-4371-4ae7-b920-251834e07877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:23.475808 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2113c46e-4371-4ae7-b920-251834e07877 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:23.487426 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1145/2298] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:24:23.491944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a3ef2-2fda-47a4-9b89-8513fba24f3e None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:23.506161 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a3ef2-2fda-47a4-9b89-8513fba24f3e None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:23.514213 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c6bc8-87b7-4e65-9fd6-bcc8856393a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:23.515075 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c6bc8-87b7-4e65-9fd6-bcc8856393a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:23.515463 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c6bc8-87b7-4e65-9fd6-bcc8856393a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:23.515839 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c6bc8-87b7-4e65-9fd6-bcc8856393a5 admin admin] PATH_INFO: `/v3/projects/ef2d76764a43493fabfdde4b189c4511/users/fdd3b187d3e94485b5d7ff8a353c001c/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:23.540042 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c6bc8-87b7-4e65-9fd6-bcc8856393a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef2d76764a43493fabfdde4b189c4511, user_id=fdd3b187d3e94485b5d7ff8a353c001c, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:23.542765 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c6bc8-87b7-4e65-9fd6-bcc8856393a5 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:23.558255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1154/2299] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:24:23 2022] PUT /identity/v3/projects/ef2d76764a43493fabfdde4b189c4511/users/fdd3b187d3e94485b5d7ff8a353c001c/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:23.572078 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046668c0-d067-499a-b723-815d846e0e5b None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:23.572563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046668c0-d067-499a-b723-815d846e0e5b None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:23.572934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046668c0-d067-499a-b723-815d846e0e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:23.606280 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-046668c0-d067-499a-b723-815d846e0e5b None None] MFA Rules not processed for user `fdd3b187d3e94485b5d7ff8a353c001c`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:23.608115 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-046668c0-d067-499a-b723-815d846e0e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:23.699392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1146/2300] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:23.711465 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c6bc8-87b7-4e65-9fd6-bcc8856393a5 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:23.713376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-cb5c6bc8-87b7-4e65-9fd6-bcc8856393a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:23.781270 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c6bc8-87b7-4e65-9fd6-bcc8856393a5 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:23.786698 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db73e8e-42f9-4340-9c29-4431681492e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:23.787336 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db73e8e-42f9-4340-9c29-4431681492e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:23.787617 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db73e8e-42f9-4340-9c29-4431681492e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:23.787889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db73e8e-42f9-4340-9c29-4431681492e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:23.797268 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db73e8e-42f9-4340-9c29-4431681492e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:23.799415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db73e8e-42f9-4340-9c29-4431681492e7 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:23.808602 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1155/2301] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:24:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:24:23.834126 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046668c0-d067-499a-b723-815d846e0e5b None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:23.880900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:23.929862 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046668c0-d067-499a-b723-815d846e0e5b None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:23.935527 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8862890-fd46-4721-b417-a0b5adba2e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:23.936172 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8862890-fd46-4721-b417-a0b5adba2e5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:23.936453 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8862890-fd46-4721-b417-a0b5adba2e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:23.936735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8862890-fd46-4721-b417-a0b5adba2e5b admin admin] PATH_INFO: `/v3/users/fdd3b187d3e94485b5d7ff8a353c001c` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:23.938243 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8862890-fd46-4721-b417-a0b5adba2e5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdd3b187d3e94485b5d7ff8a353c001c)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:23.940694 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8862890-fd46-4721-b417-a0b5adba2e5b admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:24.010144 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-a8862890-fd46-4721-b417-a0b5adba2e5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdd3b187d3e94485b5d7ff8a353c001c'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:24.027086 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-a8862890-fd46-4721-b417-a0b5adba2e5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdd3b187d3e94485b5d7ff8a353c001c'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:24.035582 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-a8862890-fd46-4721-b417-a0b5adba2e5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdd3b187d3e94485b5d7ff8a353c001c'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:24.045108 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-a8862890-fd46-4721-b417-a0b5adba2e5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdd3b187d3e94485b5d7ff8a353c001c'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:24.046874 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-a8862890-fd46-4721-b417-a0b5adba2e5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdd3b187d3e94485b5d7ff8a353c001c'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:24.058341 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1147/2302] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:24:23 2022] DELETE /identity/v3/users/fdd3b187d3e94485b5d7ff8a353c001c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:24.062921 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d92093-93aa-447a-8436-b0cc0ccbb2b1 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:24.064785 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d92093-93aa-447a-8436-b0cc0ccbb2b1 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:24.065175 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d92093-93aa-447a-8436-b0cc0ccbb2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:24.104203 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-b5d92093-93aa-447a-8436-b0cc0ccbb2b1 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:24.105561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b5d92093-93aa-447a-8436-b0cc0ccbb2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:24.179743 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1156/2303] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:24.192951 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8862890-fd46-4721-b417-a0b5adba2e5b admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:24.195486 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-a8862890-fd46-4721-b417-a0b5adba2e5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:24.276664 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8862890-fd46-4721-b417-a0b5adba2e5b admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:24.281837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ee413-55d2-4176-b0e9-5672d4cbc766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:24.282541 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ee413-55d2-4176-b0e9-5672d4cbc766 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:24.282895 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ee413-55d2-4176-b0e9-5672d4cbc766 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:24.283299 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ee413-55d2-4176-b0e9-5672d4cbc766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:24.294960 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ee413-55d2-4176-b0e9-5672d4cbc766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:24.296606 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ee413-55d2-4176-b0e9-5672d4cbc766 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:24.307555 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1148/2304] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:24:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:24:24.548915 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d92093-93aa-447a-8436-b0cc0ccbb2b1 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:24.550991 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b5d92093-93aa-447a-8436-b0cc0ccbb2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:24.566332 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d92093-93aa-447a-8436-b0cc0ccbb2b1 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:24.576556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb43a69-2bbd-452d-a8c7-dd081cc43648 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:24.577235 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb43a69-2bbd-452d-a8c7-dd081cc43648 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:24.577470 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb43a69-2bbd-452d-a8c7-dd081cc43648 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:24.577783 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb43a69-2bbd-452d-a8c7-dd081cc43648 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:24.579601 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-bcb43a69-2bbd-452d-a8c7-dd081cc43648 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:24.623970 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ee413-55d2-4176-b0e9-5672d4cbc766 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:24.625475 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-589ee413-55d2-4176-b0e9-5672d4cbc766 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:24.625817 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:24.650232 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ee413-55d2-4176-b0e9-5672d4cbc766 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:24.655231 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7ad91-5515-4ab6-95d1-1a325f28e33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:24.655876 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7ad91-5515-4ab6-95d1-1a325f28e33b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:24.656160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7ad91-5515-4ab6-95d1-1a325f28e33b admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:24.656438 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7ad91-5515-4ab6-95d1-1a325f28e33b admin admin] PATH_INFO: `/v3/projects/ef2d76764a43493fabfdde4b189c4511` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:24.657744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b7ad91-5515-4ab6-95d1-1a325f28e33b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef2d76764a43493fabfdde4b189c4511)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:24.661568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b7ad91-5515-4ab6-95d1-1a325f28e33b admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:24.661750 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb43a69-2bbd-452d-a8c7-dd081cc43648 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:24.663282 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb43a69-2bbd-452d-a8c7-dd081cc43648 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:24.674491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1157/2305] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:24:24 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:24:24.720500 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:24.722786 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-94b7ad91-5515-4ab6-95d1-1a325f28e33b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef2d76764a43493fabfdde4b189c4511', 'request_id': 'req-94b7ad91-5515-4ab6-95d1-1a325f28e33b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:24.731899 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-94b7ad91-5515-4ab6-95d1-1a325f28e33b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef2d76764a43493fabfdde4b189c4511', 'request_id': 'req-94b7ad91-5515-4ab6-95d1-1a325f28e33b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:24.740858 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-94b7ad91-5515-4ab6-95d1-1a325f28e33b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef2d76764a43493fabfdde4b189c4511', 'request_id': 'req-94b7ad91-5515-4ab6-95d1-1a325f28e33b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:24.754612 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-94b7ad91-5515-4ab6-95d1-1a325f28e33b admin admin] The token cache is being invalidate because project ef2d76764a43493fabfdde4b189c4511 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:24:24.754907 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-94b7ad91-5515-4ab6-95d1-1a325f28e33b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:24.755650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-94b7ad91-5515-4ab6-95d1-1a325f28e33b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:24.757422 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1149/2306] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:24:24 2022] DELETE /identity/v3/projects/ef2d76764a43493fabfdde4b189c4511 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:24:24.762237 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179f693-ef5c-4056-be0b-11fafd25520d None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:24.762530 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179f693-ef5c-4056-be0b-11fafd25520d None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:24.762815 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179f693-ef5c-4056-be0b-11fafd25520d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:24.786626 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-4179f693-ef5c-4056-be0b-11fafd25520d None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:24.788040 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4179f693-ef5c-4056-be0b-11fafd25520d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:24.854628 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1158/2307] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:24.863350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12a4c1-a585-442d-be14-3c726b43672e None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:24.863851 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12a4c1-a585-442d-be14-3c726b43672e None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:24.864314 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12a4c1-a585-442d-be14-3c726b43672e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:24.888221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-aa12a4c1-a585-442d-be14-3c726b43672e None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:24.889466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-aa12a4c1-a585-442d-be14-3c726b43672e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:24.900179 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1150/2308] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:24.904519 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4179f693-ef5c-4056-be0b-11fafd25520d None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:24.915752 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4179f693-ef5c-4056-be0b-11fafd25520d None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:24.920802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1e878-13e8-4348-a058-c1f8db6ee3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:24.921416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1e878-13e8-4348-a058-c1f8db6ee3e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:24.921695 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1e878-13e8-4348-a058-c1f8db6ee3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:24.921967 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1e878-13e8-4348-a058-c1f8db6ee3e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:24.922675 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1e878-13e8-4348-a058-c1f8db6ee3e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:24:24.922982 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1e878-13e8-4348-a058-c1f8db6ee3e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:24.924915 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1e878-13e8-4348-a058-c1f8db6ee3e8 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:24.937905 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1159/2309] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:24:24.943603 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12a4c1-a585-442d-be14-3c726b43672e None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:24.954839 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12a4c1-a585-442d-be14-3c726b43672e None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:24.960442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61caf3-920f-491b-a8da-fdddebc31bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:24.961180 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61caf3-920f-491b-a8da-fdddebc31bfb admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:24.961617 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61caf3-920f-491b-a8da-fdddebc31bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:24.961895 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61caf3-920f-491b-a8da-fdddebc31bfb admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:24.962548 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61caf3-920f-491b-a8da-fdddebc31bfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:24.964942 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61caf3-920f-491b-a8da-fdddebc31bfb admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:25.001606 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1151/2310] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:24:24 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:24:25.006737 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1e878-13e8-4348-a058-c1f8db6ee3e8 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:25.019004 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1e878-13e8-4348-a058-c1f8db6ee3e8 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:25.082448 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db0df7-9313-4acb-beb8-0e7241681111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:25.083257 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db0df7-9313-4acb-beb8-0e7241681111 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:25.083618 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db0df7-9313-4acb-beb8-0e7241681111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:25.083973 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db0df7-9313-4acb-beb8-0e7241681111 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:25.084683 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db0df7-9313-4acb-beb8-0e7241681111 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:25.086953 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db0df7-9313-4acb-beb8-0e7241681111 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:25.119108 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1160/2311] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:24:25 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:24:25.124147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61caf3-920f-491b-a8da-fdddebc31bfb admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:25.136753 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61caf3-920f-491b-a8da-fdddebc31bfb admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:25.141516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33932b5d-7e5e-4cab-8189-4d5576ef834a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:25.143519 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33932b5d-7e5e-4cab-8189-4d5576ef834a admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:25.143519 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33932b5d-7e5e-4cab-8189-4d5576ef834a admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:25.143519 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33932b5d-7e5e-4cab-8189-4d5576ef834a admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:25.144604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33932b5d-7e5e-4cab-8189-4d5576ef834a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:25.145972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33932b5d-7e5e-4cab-8189-4d5576ef834a admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:25.156618 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1152/2312] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:24:25.162626 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db0df7-9313-4acb-beb8-0e7241681111 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:25.180695 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db0df7-9313-4acb-beb8-0e7241681111 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:25.190720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94381f6c-a2ec-4272-8c95-50a765826136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:25.191495 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94381f6c-a2ec-4272-8c95-50a765826136 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:25.191775 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94381f6c-a2ec-4272-8c95-50a765826136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:25.192051 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94381f6c-a2ec-4272-8c95-50a765826136 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:25.192713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94381f6c-a2ec-4272-8c95-50a765826136 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:25.194379 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94381f6c-a2ec-4272-8c95-50a765826136 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:25.205995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1161/2313] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:24:25.209944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33932b5d-7e5e-4cab-8189-4d5576ef834a admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:25.220421 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33932b5d-7e5e-4cab-8189-4d5576ef834a admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:25.226122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1909b4-b140-4db0-8f78-9cb08ddaeb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:25.226798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1909b4-b140-4db0-8f78-9cb08ddaeb05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:25.227108 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1909b4-b140-4db0-8f78-9cb08ddaeb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:25.227390 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1909b4-b140-4db0-8f78-9cb08ddaeb05 admin admin] PATH_INFO: `/v3/projects/ec698ab0ef6641f48149376bf0229f04/users/9960b60968074aee9bdf32218865d696/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:25.243667 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1909b4-b140-4db0-8f78-9cb08ddaeb05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec698ab0ef6641f48149376bf0229f04, user_id=9960b60968074aee9bdf32218865d696, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:25.246375 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1909b4-b140-4db0-8f78-9cb08ddaeb05 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:25.260843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1153/2314] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:24:25 2022] PUT /identity/v3/projects/ec698ab0ef6641f48149376bf0229f04/users/9960b60968074aee9bdf32218865d696/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:25.272372 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfec2b6-5a30-4d46-9b8e-82363acc4069 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:25.272657 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfec2b6-5a30-4d46-9b8e-82363acc4069 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:25.272931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfec2b6-5a30-4d46-9b8e-82363acc4069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:25.304250 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-8bfec2b6-5a30-4d46-9b8e-82363acc4069 None None] MFA Rules not processed for user `9960b60968074aee9bdf32218865d696`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:25.305487 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-8bfec2b6-5a30-4d46-9b8e-82363acc4069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:25.335968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:25.381857 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1162/2315] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:25.392863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1909b4-b140-4db0-8f78-9cb08ddaeb05 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:25.394988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-dd1909b4-b140-4db0-8f78-9cb08ddaeb05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:25.483511 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1909b4-b140-4db0-8f78-9cb08ddaeb05 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:25.487944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5c269-b7c5-480d-8bbc-012212932f76 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55714a21c1cb4f2b949603230243ed02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:25.488559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5c269-b7c5-480d-8bbc-012212932f76 service glance] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:25.488838 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5c269-b7c5-480d-8bbc-012212932f76 service glance] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:25.489110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5c269-b7c5-480d-8bbc-012212932f76 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:25.498739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d5c269-b7c5-480d-8bbc-012212932f76 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:25.500234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d5c269-b7c5-480d-8bbc-012212932f76 service glance] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:25.512293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1154/2316] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:24:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:24:35.378803 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=14 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:35.507570 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=69,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:56.966565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfec2b6-5a30-4d46-9b8e-82363acc4069 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:57.028289 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfec2b6-5a30-4d46-9b8e-82363acc4069 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:57.032673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95170699-102e-4107-b235-b54d02648ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:57.033303 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95170699-102e-4107-b235-b54d02648ffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:57.033579 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95170699-102e-4107-b235-b54d02648ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:57.033845 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95170699-102e-4107-b235-b54d02648ffd admin admin] PATH_INFO: `/v3/users/9960b60968074aee9bdf32218865d696` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:57.035532 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95170699-102e-4107-b235-b54d02648ffd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9960b60968074aee9bdf32218865d696)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:57.037458 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95170699-102e-4107-b235-b54d02648ffd admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:57.083141 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-95170699-102e-4107-b235-b54d02648ffd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9960b60968074aee9bdf32218865d696'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:57.090525 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-95170699-102e-4107-b235-b54d02648ffd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9960b60968074aee9bdf32218865d696'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:57.097730 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-95170699-102e-4107-b235-b54d02648ffd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9960b60968074aee9bdf32218865d696'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:57.098508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-95170699-102e-4107-b235-b54d02648ffd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9960b60968074aee9bdf32218865d696'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:57.099198 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-95170699-102e-4107-b235-b54d02648ffd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9960b60968074aee9bdf32218865d696'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:57.117332 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1163/2317] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:24:56 2022] DELETE /identity/v3/users/9960b60968074aee9bdf32218865d696 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:57.121764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891d7e3-11a5-46ce-9179-af0ff29e8442 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:57.122271 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891d7e3-11a5-46ce-9179-af0ff29e8442 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:57.122375 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891d7e3-11a5-46ce-9179-af0ff29e8442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:57.143115 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-3891d7e3-11a5-46ce-9179-af0ff29e8442 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:57.144459 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-3891d7e3-11a5-46ce-9179-af0ff29e8442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:57.201877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1155/2318] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:57.212132 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95170699-102e-4107-b235-b54d02648ffd admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:57.214107 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-95170699-102e-4107-b235-b54d02648ffd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:57.285306 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95170699-102e-4107-b235-b54d02648ffd admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:57.288970 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31525bd3-c8e7-4c67-8cc7-d74fa6c16347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:57.289604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31525bd3-c8e7-4c67-8cc7-d74fa6c16347 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:57.289946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31525bd3-c8e7-4c67-8cc7-d74fa6c16347 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:57.290260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31525bd3-c8e7-4c67-8cc7-d74fa6c16347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:57.299788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31525bd3-c8e7-4c67-8cc7-d74fa6c16347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:57.301895 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31525bd3-c8e7-4c67-8cc7-d74fa6c16347 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:57.311567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1164/2319] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:24:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:24:57.809997 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891d7e3-11a5-46ce-9179-af0ff29e8442 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:57.811465 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-3891d7e3-11a5-46ce-9179-af0ff29e8442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:57.825349 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891d7e3-11a5-46ce-9179-af0ff29e8442 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:57.829992 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf2eb2-7918-4642-9b81-95c991c58580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:57.831340 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf2eb2-7918-4642-9b81-95c991c58580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:57.832497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf2eb2-7918-4642-9b81-95c991c58580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:57.832685 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf2eb2-7918-4642-9b81-95c991c58580 admin admin] PATH_INFO: `/v3/projects/ec698ab0ef6641f48149376bf0229f04` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:57.833833 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf2eb2-7918-4642-9b81-95c991c58580 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec698ab0ef6641f48149376bf0229f04)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:57.835799 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf2eb2-7918-4642-9b81-95c991c58580 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:57.884970 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-d8cf2eb2-7918-4642-9b81-95c991c58580 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec698ab0ef6641f48149376bf0229f04', 'request_id': 'req-d8cf2eb2-7918-4642-9b81-95c991c58580'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:57.892599 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-d8cf2eb2-7918-4642-9b81-95c991c58580 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec698ab0ef6641f48149376bf0229f04', 'request_id': 'req-d8cf2eb2-7918-4642-9b81-95c991c58580'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:57.899137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-d8cf2eb2-7918-4642-9b81-95c991c58580 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec698ab0ef6641f48149376bf0229f04', 'request_id': 'req-d8cf2eb2-7918-4642-9b81-95c991c58580'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:57.913142 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-d8cf2eb2-7918-4642-9b81-95c991c58580 admin admin] The token cache is being invalidate because project ec698ab0ef6641f48149376bf0229f04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:24:57.913445 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-d8cf2eb2-7918-4642-9b81-95c991c58580 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:57.914108 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-d8cf2eb2-7918-4642-9b81-95c991c58580 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:24:57.915640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1156/2320] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:24:57 2022] DELETE /identity/v3/projects/ec698ab0ef6641f48149376bf0229f04 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:57.920880 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b614db-eede-4859-bb9f-8a2f42ae45ca None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:57.921118 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b614db-eede-4859-bb9f-8a2f42ae45ca None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:57.921394 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b614db-eede-4859-bb9f-8a2f42ae45ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:57.940896 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-a5b614db-eede-4859-bb9f-8a2f42ae45ca None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:57.942693 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a5b614db-eede-4859-bb9f-8a2f42ae45ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:57.960248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:58.005181 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1165/2321] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:58.012828 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26050c41-06dd-4981-8cf0-d2befecb41d9 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:58.013104 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26050c41-06dd-4981-8cf0-d2befecb41d9 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:58.013382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26050c41-06dd-4981-8cf0-d2befecb41d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:58.033666 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-26050c41-06dd-4981-8cf0-d2befecb41d9 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:58.034880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-26050c41-06dd-4981-8cf0-d2befecb41d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:58.042720 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1157/2322] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:58.046951 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b614db-eede-4859-bb9f-8a2f42ae45ca None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:58.056960 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b614db-eede-4859-bb9f-8a2f42ae45ca None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:58.061604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79975403-dd6f-4cc3-aefc-95518bd5085d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:58.062204 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79975403-dd6f-4cc3-aefc-95518bd5085d admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:58.062483 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79975403-dd6f-4cc3-aefc-95518bd5085d admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:58.062756 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79975403-dd6f-4cc3-aefc-95518bd5085d admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:58.063536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79975403-dd6f-4cc3-aefc-95518bd5085d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:24:58.063844 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79975403-dd6f-4cc3-aefc-95518bd5085d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:58.065445 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79975403-dd6f-4cc3-aefc-95518bd5085d admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:58.077428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1166/2323] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:24:58.082457 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26050c41-06dd-4981-8cf0-d2befecb41d9 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:58.092299 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26050c41-06dd-4981-8cf0-d2befecb41d9 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:58.096977 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411feaaf-5e87-4228-9aaf-fff5401af4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:58.097814 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411feaaf-5e87-4228-9aaf-fff5401af4db admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:58.098085 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411feaaf-5e87-4228-9aaf-fff5401af4db admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:58.098355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411feaaf-5e87-4228-9aaf-fff5401af4db admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:58.098978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411feaaf-5e87-4228-9aaf-fff5401af4db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:58.100992 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411feaaf-5e87-4228-9aaf-fff5401af4db admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:58.129323 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1158/2324] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:24:58 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:24:58.133706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79975403-dd6f-4cc3-aefc-95518bd5085d admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:58.143992 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79975403-dd6f-4cc3-aefc-95518bd5085d admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:58.201751 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788b597-c8ff-4450-8356-2a3f834955c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:58.202426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788b597-c8ff-4450-8356-2a3f834955c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:58.202706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788b597-c8ff-4450-8356-2a3f834955c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:58.202979 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788b597-c8ff-4450-8356-2a3f834955c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:58.203668 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788b597-c8ff-4450-8356-2a3f834955c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:58.205743 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788b597-c8ff-4450-8356-2a3f834955c4 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:58.228253 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1167/2325] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:24:58 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:24:58.232330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411feaaf-5e87-4228-9aaf-fff5401af4db admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:58.243270 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411feaaf-5e87-4228-9aaf-fff5401af4db admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:58.247884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46057a83-0d6e-4887-b15d-3db84d6e2712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:58.248484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46057a83-0d6e-4887-b15d-3db84d6e2712 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:58.248751 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46057a83-0d6e-4887-b15d-3db84d6e2712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:58.249020 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46057a83-0d6e-4887-b15d-3db84d6e2712 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:58.249654 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46057a83-0d6e-4887-b15d-3db84d6e2712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:58.251279 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46057a83-0d6e-4887-b15d-3db84d6e2712 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:58.261453 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1159/2326] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:24:58.266081 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788b597-c8ff-4450-8356-2a3f834955c4 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:58.278684 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788b597-c8ff-4450-8356-2a3f834955c4 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:58.283765 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d5805-252a-4127-bfe3-0c4d734cfb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:58.284362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d5805-252a-4127-bfe3-0c4d734cfb11 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:58.284646 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d5805-252a-4127-bfe3-0c4d734cfb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:58.285564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d5805-252a-4127-bfe3-0c4d734cfb11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:58.286368 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d5805-252a-4127-bfe3-0c4d734cfb11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:58.288011 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d5805-252a-4127-bfe3-0c4d734cfb11 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:58.299907 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1168/2327] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:24:58.304551 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46057a83-0d6e-4887-b15d-3db84d6e2712 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:58.314792 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46057a83-0d6e-4887-b15d-3db84d6e2712 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:58.319993 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396a7dc-816f-459b-bf40-aea0c221adaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:58.320676 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396a7dc-816f-459b-bf40-aea0c221adaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:58.320985 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396a7dc-816f-459b-bf40-aea0c221adaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:58.321264 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396a7dc-816f-459b-bf40-aea0c221adaf admin admin] PATH_INFO: `/v3/projects/f18433570ad14f85be4a98f94a2aed7b/users/dc274866b31f4d2ea3ced9990c8869d0/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:58.336864 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396a7dc-816f-459b-bf40-aea0c221adaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f18433570ad14f85be4a98f94a2aed7b, user_id=dc274866b31f4d2ea3ced9990c8869d0, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:58.339582 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396a7dc-816f-459b-bf40-aea0c221adaf admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:58.353054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1160/2328] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:24:58 2022] PUT /identity/v3/projects/f18433570ad14f85be4a98f94a2aed7b/users/dc274866b31f4d2ea3ced9990c8869d0/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:58.363868 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9cdb8f-1811-4221-b6fc-31ce61d06312 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:58.364159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9cdb8f-1811-4221-b6fc-31ce61d06312 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:58.364432 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9cdb8f-1811-4221-b6fc-31ce61d06312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:58.391585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-4a9cdb8f-1811-4221-b6fc-31ce61d06312 None None] MFA Rules not processed for user `dc274866b31f4d2ea3ced9990c8869d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:58.392826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4a9cdb8f-1811-4221-b6fc-31ce61d06312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:58.458220 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1169/2329] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:24:58.462878 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396a7dc-816f-459b-bf40-aea0c221adaf admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:58.483813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:58.526455 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396a7dc-816f-459b-bf40-aea0c221adaf admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:58.531104 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8067e1-efed-49c5-a79d-bd7c89b3f7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:58.531713 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8067e1-efed-49c5-a79d-bd7c89b3f7c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:58.531992 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8067e1-efed-49c5-a79d-bd7c89b3f7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:58.532265 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8067e1-efed-49c5-a79d-bd7c89b3f7c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:58.532904 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8067e1-efed-49c5-a79d-bd7c89b3f7c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:58.535695 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8067e1-efed-49c5-a79d-bd7c89b3f7c6 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:58.561859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1161/2330] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:24:58 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:24:58.566080 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9cdb8f-1811-4221-b6fc-31ce61d06312 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:58.578028 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9cdb8f-1811-4221-b6fc-31ce61d06312 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:58.591379 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:58.637789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bf238c-d88b-494b-971f-fab9df5098b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:58.638429 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf238c-d88b-494b-971f-fab9df5098b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:58.638713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf238c-d88b-494b-971f-fab9df5098b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:58.638985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf238c-d88b-494b-971f-fab9df5098b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:58.639651 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bf238c-d88b-494b-971f-fab9df5098b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:58.641596 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bf238c-d88b-494b-971f-fab9df5098b5 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:58.664595 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1170/2331] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:24:58 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:24:58.668756 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8067e1-efed-49c5-a79d-bd7c89b3f7c6 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:58.685643 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8067e1-efed-49c5-a79d-bd7c89b3f7c6 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:58.690378 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6672141-0044-449b-be39-d4f331d7ba80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:58.690998 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6672141-0044-449b-be39-d4f331d7ba80 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:58.691311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6672141-0044-449b-be39-d4f331d7ba80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:58.691588 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6672141-0044-449b-be39-d4f331d7ba80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:58.692801 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6672141-0044-449b-be39-d4f331d7ba80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:58.694415 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6672141-0044-449b-be39-d4f331d7ba80 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:58.705812 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1162/2332] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:24:58.711706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bf238c-d88b-494b-971f-fab9df5098b5 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:58.722545 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bf238c-d88b-494b-971f-fab9df5098b5 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:58.727677 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becabdf9-3f16-4c36-a119-100a0b55a789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:58.728286 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becabdf9-3f16-4c36-a119-100a0b55a789 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:58.728568 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becabdf9-3f16-4c36-a119-100a0b55a789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:58.728845 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becabdf9-3f16-4c36-a119-100a0b55a789 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:58.729484 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becabdf9-3f16-4c36-a119-100a0b55a789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:58.731130 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becabdf9-3f16-4c36-a119-100a0b55a789 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:58.741540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1171/2333] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:24:58.745720 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6672141-0044-449b-be39-d4f331d7ba80 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:58.756895 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6672141-0044-449b-be39-d4f331d7ba80 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:58.761450 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecfa1a-187b-4d60-9e2a-e3ec4df447d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:58.762312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecfa1a-187b-4d60-9e2a-e3ec4df447d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:58.762758 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecfa1a-187b-4d60-9e2a-e3ec4df447d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:58.763070 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecfa1a-187b-4d60-9e2a-e3ec4df447d1 admin admin] PATH_INFO: `/v3/projects/99fc31c944494a0b9510e54033c7829c/users/583e8d4698574a41ba149646e24cc7cb/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:58.778650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ecfa1a-187b-4d60-9e2a-e3ec4df447d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99fc31c944494a0b9510e54033c7829c, user_id=583e8d4698574a41ba149646e24cc7cb, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:58.781299 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ecfa1a-187b-4d60-9e2a-e3ec4df447d1 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:58.794072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1163/2334] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:24:58 2022] PUT /identity/v3/projects/99fc31c944494a0b9510e54033c7829c/users/583e8d4698574a41ba149646e24cc7cb/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:58.805865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a2a90-09ea-4978-b91e-56a289ad0211 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:58.806147 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a2a90-09ea-4978-b91e-56a289ad0211 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:58.806421 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a2a90-09ea-4978-b91e-56a289ad0211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:58.834512 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-ac0a2a90-09ea-4978-b91e-56a289ad0211 None None] MFA Rules not processed for user `583e8d4698574a41ba149646e24cc7cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:58.835730 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-ac0a2a90-09ea-4978-b91e-56a289ad0211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:58.898222 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1172/2335] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:58.902896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecfa1a-187b-4d60-9e2a-e3ec4df447d1 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:58.965385 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecfa1a-187b-4d60-9e2a-e3ec4df447d1 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:58.970173 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e3018-c5b1-45c3-8c4d-6151a8832839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:58.970784 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e3018-c5b1-45c3-8c4d-6151a8832839 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:58.971096 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e3018-c5b1-45c3-8c4d-6151a8832839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:58.971373 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e3018-c5b1-45c3-8c4d-6151a8832839 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:58.972037 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e3018-c5b1-45c3-8c4d-6151a8832839 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:58.974504 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e3018-c5b1-45c3-8c4d-6151a8832839 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:59.010862 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1164/2336] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:24:58 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:24:59.012534 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a2a90-09ea-4978-b91e-56a289ad0211 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:59.024174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a2a90-09ea-4978-b91e-56a289ad0211 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:59.082496 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5fe3e-6f16-495a-81bb-ed461f937184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:59.083134 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5fe3e-6f16-495a-81bb-ed461f937184 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:59.083420 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5fe3e-6f16-495a-81bb-ed461f937184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:59.083697 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5fe3e-6f16-495a-81bb-ed461f937184 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:59.084345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f5fe3e-6f16-495a-81bb-ed461f937184 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:59.086240 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f5fe3e-6f16-495a-81bb-ed461f937184 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:59.108813 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1173/2337] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:24:59 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:24:59.112870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e3018-c5b1-45c3-8c4d-6151a8832839 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:59.124462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e3018-c5b1-45c3-8c4d-6151a8832839 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:59.129446 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e942b6-f7ea-4f53-b94d-9825a93d1036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:59.130711 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e942b6-f7ea-4f53-b94d-9825a93d1036 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:59.131000 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e942b6-f7ea-4f53-b94d-9825a93d1036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:59.131303 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e942b6-f7ea-4f53-b94d-9825a93d1036 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:59.131951 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e942b6-f7ea-4f53-b94d-9825a93d1036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:59.133558 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e942b6-f7ea-4f53-b94d-9825a93d1036 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:59.144649 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1165/2338] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:24:59.149184 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5fe3e-6f16-495a-81bb-ed461f937184 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:59.159217 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5fe3e-6f16-495a-81bb-ed461f937184 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:59.163962 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac80407-3933-4362-9f44-eb8d80149954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:59.164604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac80407-3933-4362-9f44-eb8d80149954 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:59.164885 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac80407-3933-4362-9f44-eb8d80149954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:59.165153 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac80407-3933-4362-9f44-eb8d80149954 admin admin] PATH_INFO: `/v3/projects/cff211fcacc248938f8210671ba613d4/users/22dbf1c676ef48cabae5e5e9a0fdc277/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:59.177819 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:59.181996 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac80407-3933-4362-9f44-eb8d80149954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cff211fcacc248938f8210671ba613d4, user_id=22dbf1c676ef48cabae5e5e9a0fdc277, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:59.185266 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac80407-3933-4362-9f44-eb8d80149954 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:59.202749 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1174/2339] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:24:59 2022] PUT /identity/v3/projects/cff211fcacc248938f8210671ba613d4/users/22dbf1c676ef48cabae5e5e9a0fdc277/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:59.204826 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e942b6-f7ea-4f53-b94d-9825a93d1036 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:59.216595 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e942b6-f7ea-4f53-b94d-9825a93d1036 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:59.261010 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:24:59.279004 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fa4e3-9f7b-4932-bded-069c58c9755e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:59.279690 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fa4e3-9f7b-4932-bded-069c58c9755e admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:59.279980 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fa4e3-9f7b-4932-bded-069c58c9755e admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:59.280278 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fa4e3-9f7b-4932-bded-069c58c9755e admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:59.280896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8fa4e3-9f7b-4932-bded-069c58c9755e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:59.282488 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8fa4e3-9f7b-4932-bded-069c58c9755e admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:59.293764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1166/2340] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:24:59.300368 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac80407-3933-4362-9f44-eb8d80149954 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:59.317597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac80407-3933-4362-9f44-eb8d80149954 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:59.322567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361ea30-9340-4621-b77c-32a9cea43ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:59.323399 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361ea30-9340-4621-b77c-32a9cea43ea1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:59.323746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361ea30-9340-4621-b77c-32a9cea43ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:59.324088 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361ea30-9340-4621-b77c-32a9cea43ea1 admin admin] PATH_INFO: `/v3/projects/cff211fcacc248938f8210671ba613d4/users/22dbf1c676ef48cabae5e5e9a0fdc277/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:59.326090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361ea30-9340-4621-b77c-32a9cea43ea1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cff211fcacc248938f8210671ba613d4, user_id=22dbf1c676ef48cabae5e5e9a0fdc277, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:59.329093 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361ea30-9340-4621-b77c-32a9cea43ea1 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:59.340980 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1175/2341] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:24:59 2022] PUT /identity/v3/projects/cff211fcacc248938f8210671ba613d4/users/22dbf1c676ef48cabae5e5e9a0fdc277/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:24:59.351798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0b92f-8fef-4421-9d7f-444990972933 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:59.352302 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0b92f-8fef-4421-9d7f-444990972933 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:59.352611 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0b92f-8fef-4421-9d7f-444990972933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:59.386831 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-24e0b92f-8fef-4421-9d7f-444990972933 None None] MFA Rules not processed for user `22dbf1c676ef48cabae5e5e9a0fdc277`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:24:59.388061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-24e0b92f-8fef-4421-9d7f-444990972933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:59.457517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1167/2342] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:24:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:24:59.469346 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361ea30-9340-4621-b77c-32a9cea43ea1 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:24:59.470978 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-2361ea30-9340-4621-b77c-32a9cea43ea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:24:59.528489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361ea30-9340-4621-b77c-32a9cea43ea1 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:24:59.532155 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0937f9-f3ec-4332-a80a-650ddb142480 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:24:59.532762 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0937f9-f3ec-4332-a80a-650ddb142480 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:24:59.533044 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0937f9-f3ec-4332-a80a-650ddb142480 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:24:59.533320 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0937f9-f3ec-4332-a80a-650ddb142480 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:24:59.593684 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0937f9-f3ec-4332-a80a-650ddb142480 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:24:59.595153 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0937f9-f3ec-4332-a80a-650ddb142480 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:24:59.604953 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1176/2343] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:24:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:25:03.244914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0b92f-8fef-4421-9d7f-444990972933 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:03.246361 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-24e0b92f-8fef-4421-9d7f-444990972933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:03.260456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0b92f-8fef-4421-9d7f-444990972933 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:03.264859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f5f41-8315-497d-9f70-905228384c34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:03.265421 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f5f41-8315-497d-9f70-905228384c34 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:03.265706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f5f41-8315-497d-9f70-905228384c34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:03.266153 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f5f41-8315-497d-9f70-905228384c34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:03.335398 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f5f41-8315-497d-9f70-905228384c34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:03.336835 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f5f41-8315-497d-9f70-905228384c34 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:03.345688 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1168/2344] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:25:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:25:03.896598 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0937f9-f3ec-4332-a80a-650ddb142480 service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:03.905305 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0937f9-f3ec-4332-a80a-650ddb142480 service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:03.908995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19017fe-a7b6-4d8c-b94c-57af767e4e3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:03.909594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19017fe-a7b6-4d8c-b94c-57af767e4e3b service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:03.909881 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19017fe-a7b6-4d8c-b94c-57af767e4e3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:03.910157 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19017fe-a7b6-4d8c-b94c-57af767e4e3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:03.920362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19017fe-a7b6-4d8c-b94c-57af767e4e3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:03.921808 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19017fe-a7b6-4d8c-b94c-57af767e4e3b service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:03.931974 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1177/2345] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:25:03 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:25:10.012923 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f5f41-8315-497d-9f70-905228384c34 service cinder] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:10.086422 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f5f41-8315-497d-9f70-905228384c34 service cinder] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:10.091467 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5df38-722e-46d5-8e21-88368b72d7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:10.093369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5df38-722e-46d5-8e21-88368b72d7ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:10.093369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5df38-722e-46d5-8e21-88368b72d7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:10.093369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5df38-722e-46d5-8e21-88368b72d7ee admin admin] PATH_INFO: `/v3/users/dc274866b31f4d2ea3ced9990c8869d0` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:10.094158 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5df38-722e-46d5-8e21-88368b72d7ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc274866b31f4d2ea3ced9990c8869d0)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:10.096169 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5df38-722e-46d5-8e21-88368b72d7ee admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:10.101158 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:25:10.144146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-63f5df38-722e-46d5-8e21-88368b72d7ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc274866b31f4d2ea3ced9990c8869d0'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:10.156936 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-63f5df38-722e-46d5-8e21-88368b72d7ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc274866b31f4d2ea3ced9990c8869d0'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:10.164040 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-63f5df38-722e-46d5-8e21-88368b72d7ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc274866b31f4d2ea3ced9990c8869d0'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:10.171279 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-63f5df38-722e-46d5-8e21-88368b72d7ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc274866b31f4d2ea3ced9990c8869d0'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:10.171899 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-63f5df38-722e-46d5-8e21-88368b72d7ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc274866b31f4d2ea3ced9990c8869d0'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:10.177941 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1169/2346] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:25:10 2022] DELETE /identity/v3/users/dc274866b31f4d2ea3ced9990c8869d0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:25:10.182478 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531d635-f3bb-4521-8e3a-216b5059ec81 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:10.182962 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531d635-f3bb-4521-8e3a-216b5059ec81 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:10.183086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531d635-f3bb-4521-8e3a-216b5059ec81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:10.202198 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-9531d635-f3bb-4521-8e3a-216b5059ec81 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:25:10.203436 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-9531d635-f3bb-4521-8e3a-216b5059ec81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:10.263643 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1178/2347] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:25:10.273665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5df38-722e-46d5-8e21-88368b72d7ee admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:10.275144 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-63f5df38-722e-46d5-8e21-88368b72d7ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:10.332331 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5df38-722e-46d5-8e21-88368b72d7ee admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:10.336273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16187a3-ee20-4d57-b944-860ea4156248 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:10.336879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16187a3-ee20-4d57-b944-860ea4156248 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:10.337163 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16187a3-ee20-4d57-b944-860ea4156248 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:10.337439 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16187a3-ee20-4d57-b944-860ea4156248 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:10.346364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16187a3-ee20-4d57-b944-860ea4156248 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:10.347830 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16187a3-ee20-4d57-b944-860ea4156248 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:10.357363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1170/2348] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:25:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:25:10.671663 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531d635-f3bb-4521-8e3a-216b5059ec81 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:10.674965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-9531d635-f3bb-4521-8e3a-216b5059ec81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:10.687621 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531d635-f3bb-4521-8e3a-216b5059ec81 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:10.692435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babaa85d-5ed0-450a-b928-5e42902958b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:10.693076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babaa85d-5ed0-450a-b928-5e42902958b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:10.693354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babaa85d-5ed0-450a-b928-5e42902958b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:10.693622 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babaa85d-5ed0-450a-b928-5e42902958b5 admin admin] PATH_INFO: `/v3/projects/f18433570ad14f85be4a98f94a2aed7b` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:10.694738 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babaa85d-5ed0-450a-b928-5e42902958b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f18433570ad14f85be4a98f94a2aed7b)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:10.696703 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babaa85d-5ed0-450a-b928-5e42902958b5 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:10.713225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:25:10.746285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-babaa85d-5ed0-450a-b928-5e42902958b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f18433570ad14f85be4a98f94a2aed7b', 'request_id': 'req-babaa85d-5ed0-450a-b928-5e42902958b5'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:10.751793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-babaa85d-5ed0-450a-b928-5e42902958b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f18433570ad14f85be4a98f94a2aed7b', 'request_id': 'req-babaa85d-5ed0-450a-b928-5e42902958b5'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:10.759101 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-babaa85d-5ed0-450a-b928-5e42902958b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f18433570ad14f85be4a98f94a2aed7b', 'request_id': 'req-babaa85d-5ed0-450a-b928-5e42902958b5'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:10.772867 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-babaa85d-5ed0-450a-b928-5e42902958b5 admin admin] The token cache is being invalidate because project f18433570ad14f85be4a98f94a2aed7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:25:10.773261 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-babaa85d-5ed0-450a-b928-5e42902958b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:10.774007 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-babaa85d-5ed0-450a-b928-5e42902958b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:10.775619 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1179/2349] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:25:10 2022] DELETE /identity/v3/projects/f18433570ad14f85be4a98f94a2aed7b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:25:10.779968 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16187a3-ee20-4d57-b944-860ea4156248 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:10.793396 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16187a3-ee20-4d57-b944-860ea4156248 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:10.854428 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7fd66f-f669-43b7-bb84-fa3bbc91a653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:10.855066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fd66f-f669-43b7-bb84-fa3bbc91a653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:10.855353 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fd66f-f669-43b7-bb84-fa3bbc91a653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:10.855621 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fd66f-f669-43b7-bb84-fa3bbc91a653 admin admin] PATH_INFO: `/v3/users/583e8d4698574a41ba149646e24cc7cb` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:10.856766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7fd66f-f669-43b7-bb84-fa3bbc91a653 admin admin] RBAC: Authorizing `identity:delete_user(user_id=583e8d4698574a41ba149646e24cc7cb)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:10.858674 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7fd66f-f669-43b7-bb84-fa3bbc91a653 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:10.887452 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:25:10.907073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ca7fd66f-f669-43b7-bb84-fa3bbc91a653 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '583e8d4698574a41ba149646e24cc7cb'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:10.919322 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ca7fd66f-f669-43b7-bb84-fa3bbc91a653 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '583e8d4698574a41ba149646e24cc7cb'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:10.926460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ca7fd66f-f669-43b7-bb84-fa3bbc91a653 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '583e8d4698574a41ba149646e24cc7cb'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:10.933949 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ca7fd66f-f669-43b7-bb84-fa3bbc91a653 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '583e8d4698574a41ba149646e24cc7cb'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:10.934592 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ca7fd66f-f669-43b7-bb84-fa3bbc91a653 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '583e8d4698574a41ba149646e24cc7cb'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:10.940651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1171/2350] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:25:10 2022] DELETE /identity/v3/users/583e8d4698574a41ba149646e24cc7cb => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:25:11.214165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babaa85d-5ed0-450a-b928-5e42902958b5 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:11.217134 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-babaa85d-5ed0-450a-b928-5e42902958b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:11.234246 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babaa85d-5ed0-450a-b928-5e42902958b5 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:11.292251 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a24916-43cf-43c7-a9b1-513068ecae64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:11.292884 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a24916-43cf-43c7-a9b1-513068ecae64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:11.293163 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a24916-43cf-43c7-a9b1-513068ecae64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:11.293434 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a24916-43cf-43c7-a9b1-513068ecae64 admin admin] PATH_INFO: `/v3/projects/99fc31c944494a0b9510e54033c7829c` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:11.294546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a24916-43cf-43c7-a9b1-513068ecae64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99fc31c944494a0b9510e54033c7829c)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:11.296498 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a24916-43cf-43c7-a9b1-513068ecae64 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:11.343377 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-69a24916-43cf-43c7-a9b1-513068ecae64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99fc31c944494a0b9510e54033c7829c', 'request_id': 'req-69a24916-43cf-43c7-a9b1-513068ecae64'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:11.348809 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-69a24916-43cf-43c7-a9b1-513068ecae64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99fc31c944494a0b9510e54033c7829c', 'request_id': 'req-69a24916-43cf-43c7-a9b1-513068ecae64'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:11.355130 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-69a24916-43cf-43c7-a9b1-513068ecae64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99fc31c944494a0b9510e54033c7829c', 'request_id': 'req-69a24916-43cf-43c7-a9b1-513068ecae64'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:11.367564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-69a24916-43cf-43c7-a9b1-513068ecae64 admin admin] The token cache is being invalidate because project 99fc31c944494a0b9510e54033c7829c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:25:11.367851 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-69a24916-43cf-43c7-a9b1-513068ecae64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:11.368473 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-69a24916-43cf-43c7-a9b1-513068ecae64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:11.369955 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1180/2351] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:25:11 2022] DELETE /identity/v3/projects/99fc31c944494a0b9510e54033c7829c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:25:11.373939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7fd66f-f669-43b7-bb84-fa3bbc91a653 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:11.375380 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-ca7fd66f-f669-43b7-bb84-fa3bbc91a653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:11.388615 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7fd66f-f669-43b7-bb84-fa3bbc91a653 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:11.444837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd8745f-2780-49a3-b8b7-dd2821412249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:11.445455 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8745f-2780-49a3-b8b7-dd2821412249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:11.445732 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8745f-2780-49a3-b8b7-dd2821412249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:11.446002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8745f-2780-49a3-b8b7-dd2821412249 admin admin] PATH_INFO: `/v3/users/22dbf1c676ef48cabae5e5e9a0fdc277` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:11.447134 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd8745f-2780-49a3-b8b7-dd2821412249 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22dbf1c676ef48cabae5e5e9a0fdc277)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:11.449042 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd8745f-2780-49a3-b8b7-dd2821412249 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:11.495144 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ffd8745f-2780-49a3-b8b7-dd2821412249 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22dbf1c676ef48cabae5e5e9a0fdc277'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:11.507388 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ffd8745f-2780-49a3-b8b7-dd2821412249 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22dbf1c676ef48cabae5e5e9a0fdc277'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:11.514358 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ffd8745f-2780-49a3-b8b7-dd2821412249 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22dbf1c676ef48cabae5e5e9a0fdc277'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:11.523566 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ffd8745f-2780-49a3-b8b7-dd2821412249 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22dbf1c676ef48cabae5e5e9a0fdc277'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:11.524195 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ffd8745f-2780-49a3-b8b7-dd2821412249 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22dbf1c676ef48cabae5e5e9a0fdc277'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:11.530169 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1172/2352] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:25:11 2022] DELETE /identity/v3/users/22dbf1c676ef48cabae5e5e9a0fdc277 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:25:12.143026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a24916-43cf-43c7-a9b1-513068ecae64 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:12.144976 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-69a24916-43cf-43c7-a9b1-513068ecae64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:12.158265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a24916-43cf-43c7-a9b1-513068ecae64 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:12.193482 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:25:12.224735 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c3f13-adae-47e4-bfe3-1f2b89f50fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:12.225632 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c3f13-adae-47e4-bfe3-1f2b89f50fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:12.225907 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c3f13-adae-47e4-bfe3-1f2b89f50fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:12.226172 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c3f13-adae-47e4-bfe3-1f2b89f50fb0 admin admin] PATH_INFO: `/v3/projects/cff211fcacc248938f8210671ba613d4` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:12.227319 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c3f13-adae-47e4-bfe3-1f2b89f50fb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cff211fcacc248938f8210671ba613d4)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:12.229359 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c3f13-adae-47e4-bfe3-1f2b89f50fb0 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:12.275588 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3c1c3f13-adae-47e4-bfe3-1f2b89f50fb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cff211fcacc248938f8210671ba613d4', 'request_id': 'req-3c1c3f13-adae-47e4-bfe3-1f2b89f50fb0'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:12.280870 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3c1c3f13-adae-47e4-bfe3-1f2b89f50fb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cff211fcacc248938f8210671ba613d4', 'request_id': 'req-3c1c3f13-adae-47e4-bfe3-1f2b89f50fb0'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:12.287518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3c1c3f13-adae-47e4-bfe3-1f2b89f50fb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cff211fcacc248938f8210671ba613d4', 'request_id': 'req-3c1c3f13-adae-47e4-bfe3-1f2b89f50fb0'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:12.302971 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3c1c3f13-adae-47e4-bfe3-1f2b89f50fb0 admin admin] The token cache is being invalidate because project cff211fcacc248938f8210671ba613d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:25:12.303430 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3c1c3f13-adae-47e4-bfe3-1f2b89f50fb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:12.304292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3c1c3f13-adae-47e4-bfe3-1f2b89f50fb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:12.306351 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1181/2353] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:25:12 2022] DELETE /identity/v3/projects/cff211fcacc248938f8210671ba613d4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:25:12.312457 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e47b26-c4ac-46a7-9f29-62e9e756a32f None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:12.312744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e47b26-c4ac-46a7-9f29-62e9e756a32f None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:12.313052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e47b26-c4ac-46a7-9f29-62e9e756a32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:12.334860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-33e47b26-c4ac-46a7-9f29-62e9e756a32f None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:25:12.337472 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-33e47b26-c4ac-46a7-9f29-62e9e756a32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:12.355987 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:25:12.398925 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1173/2354] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:25:12.466312 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cef4a-e094-4285-b83c-246c2ab8afcd None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:12.466621 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cef4a-e094-4285-b83c-246c2ab8afcd None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:12.466897 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cef4a-e094-4285-b83c-246c2ab8afcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:12.488347 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-b50cef4a-e094-4285-b83c-246c2ab8afcd None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:25:12.489559 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b50cef4a-e094-4285-b83c-246c2ab8afcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:12.498157 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1182/2355] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:25:12.502499 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e47b26-c4ac-46a7-9f29-62e9e756a32f None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:12.512258 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e47b26-c4ac-46a7-9f29-62e9e756a32f None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:12.517174 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d2254-9e8b-476b-8e3f-a45657f8a63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:12.517778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d2254-9e8b-476b-8e3f-a45657f8a63d admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:12.518057 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d2254-9e8b-476b-8e3f-a45657f8a63d admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:12.518348 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d2254-9e8b-476b-8e3f-a45657f8a63d admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:12.518984 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392d2254-9e8b-476b-8e3f-a45657f8a63d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:25:12.519311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392d2254-9e8b-476b-8e3f-a45657f8a63d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:12.521820 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392d2254-9e8b-476b-8e3f-a45657f8a63d admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:12.533892 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1174/2356] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:25:12.538942 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50cef4a-e094-4285-b83c-246c2ab8afcd None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:12.548738 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50cef4a-e094-4285-b83c-246c2ab8afcd None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:12.553591 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb049621-692a-4ebf-a100-fa248cfbabd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:12.554194 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb049621-692a-4ebf-a100-fa248cfbabd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:12.554473 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb049621-692a-4ebf-a100-fa248cfbabd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:12.554744 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb049621-692a-4ebf-a100-fa248cfbabd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:12.555421 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb049621-692a-4ebf-a100-fa248cfbabd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:12.557322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb049621-692a-4ebf-a100-fa248cfbabd3 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:12.585342 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1183/2357] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:25:12 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:25:12.589711 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d2254-9e8b-476b-8e3f-a45657f8a63d admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:12.601827 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d2254-9e8b-476b-8e3f-a45657f8a63d admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:12.660231 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d9294-509a-4567-820e-099ecfde0e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:12.660833 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d9294-509a-4567-820e-099ecfde0e8a admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:12.661109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d9294-509a-4567-820e-099ecfde0e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:12.661376 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d9294-509a-4567-820e-099ecfde0e8a admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:12.662677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d9294-509a-4567-820e-099ecfde0e8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:12.664630 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d9294-509a-4567-820e-099ecfde0e8a admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:12.685230 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1175/2358] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:25:12 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:25:12.689347 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb049621-692a-4ebf-a100-fa248cfbabd3 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:12.699719 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb049621-692a-4ebf-a100-fa248cfbabd3 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:12.704416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129aa22-0f76-4997-baf4-f9a2635833f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:12.705023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129aa22-0f76-4997-baf4-f9a2635833f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:12.705306 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129aa22-0f76-4997-baf4-f9a2635833f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:12.705585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129aa22-0f76-4997-baf4-f9a2635833f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:12.706239 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129aa22-0f76-4997-baf4-f9a2635833f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:12.707912 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129aa22-0f76-4997-baf4-f9a2635833f4 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:12.719845 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1184/2359] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:25:12.723894 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d9294-509a-4567-820e-099ecfde0e8a admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:12.734455 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d9294-509a-4567-820e-099ecfde0e8a admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:12.738992 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85073462-b3b5-4f74-b2cd-cc72384328ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:12.739612 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85073462-b3b5-4f74-b2cd-cc72384328ad admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:12.739989 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85073462-b3b5-4f74-b2cd-cc72384328ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:12.740293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85073462-b3b5-4f74-b2cd-cc72384328ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:12.740897 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85073462-b3b5-4f74-b2cd-cc72384328ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:12.742477 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85073462-b3b5-4f74-b2cd-cc72384328ad admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:12.752912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1176/2360] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:25:12.757560 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129aa22-0f76-4997-baf4-f9a2635833f4 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:12.767195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129aa22-0f76-4997-baf4-f9a2635833f4 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:12.773028 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c06d49-725e-4243-a2c8-42a382cd4d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:12.773682 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c06d49-725e-4243-a2c8-42a382cd4d86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:12.773964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c06d49-725e-4243-a2c8-42a382cd4d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:12.774238 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c06d49-725e-4243-a2c8-42a382cd4d86 admin admin] PATH_INFO: `/v3/projects/05abcf8cdc5a4def8ac0d1a5720b1f6b/users/ea546db2fc8748c990fe65ff00e493a7/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:12.794237 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c06d49-725e-4243-a2c8-42a382cd4d86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05abcf8cdc5a4def8ac0d1a5720b1f6b, user_id=ea546db2fc8748c990fe65ff00e493a7, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:12.798113 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c06d49-725e-4243-a2c8-42a382cd4d86 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:12.815733 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1185/2361] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:25:12 2022] PUT /identity/v3/projects/05abcf8cdc5a4def8ac0d1a5720b1f6b/users/ea546db2fc8748c990fe65ff00e493a7/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:25:12.826168 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a29b55-a8f1-4ac3-abee-3839fe077215 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:12.826465 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a29b55-a8f1-4ac3-abee-3839fe077215 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:12.826731 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a29b55-a8f1-4ac3-abee-3839fe077215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:12.855887 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-e6a29b55-a8f1-4ac3-abee-3839fe077215 None None] MFA Rules not processed for user `ea546db2fc8748c990fe65ff00e493a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:25:12.857097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-e6a29b55-a8f1-4ac3-abee-3839fe077215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:12.923282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1177/2362] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:25:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:25:12.927822 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c06d49-725e-4243-a2c8-42a382cd4d86 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:12.988963 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c06d49-725e-4243-a2c8-42a382cd4d86 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:12.993787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b110f-8195-4962-b659-ec97e9af730e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:12.994396 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b110f-8195-4962-b659-ec97e9af730e admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:12.994677 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b110f-8195-4962-b659-ec97e9af730e admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:12.994976 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b110f-8195-4962-b659-ec97e9af730e admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:12.995620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b110f-8195-4962-b659-ec97e9af730e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:12.997536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b110f-8195-4962-b659-ec97e9af730e admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:13.002069 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:25:13.027308 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1186/2363] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:25:12 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:25:13.031425 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a29b55-a8f1-4ac3-abee-3839fe077215 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:13.042125 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a29b55-a8f1-4ac3-abee-3839fe077215 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:13.058158 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:25:13.098516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7da50-7146-4cc3-814b-9bd03e4bd0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:13.099149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7da50-7146-4cc3-814b-9bd03e4bd0b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:13.099424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7da50-7146-4cc3-814b-9bd03e4bd0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:13.099691 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7da50-7146-4cc3-814b-9bd03e4bd0b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:13.100331 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7da50-7146-4cc3-814b-9bd03e4bd0b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:13.102222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7da50-7146-4cc3-814b-9bd03e4bd0b9 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:13.126741 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1178/2364] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:25:13 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:25:13.128084 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b110f-8195-4962-b659-ec97e9af730e admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:13.140092 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b110f-8195-4962-b659-ec97e9af730e admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:13.145186 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870629c-d1af-45b4-84b2-dc8e4e8e5535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:13.145882 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870629c-d1af-45b4-84b2-dc8e4e8e5535 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:13.147228 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870629c-d1af-45b4-84b2-dc8e4e8e5535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:13.147794 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870629c-d1af-45b4-84b2-dc8e4e8e5535 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:13.148529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870629c-d1af-45b4-84b2-dc8e4e8e5535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:13.151016 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870629c-d1af-45b4-84b2-dc8e4e8e5535 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:13.163848 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1187/2365] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:25:13.167603 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7da50-7146-4cc3-814b-9bd03e4bd0b9 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:13.179357 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7da50-7146-4cc3-814b-9bd03e4bd0b9 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:13.184577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3d61e-b7c5-414a-b03f-3decde43d3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:13.185273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3d61e-b7c5-414a-b03f-3decde43d3d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:13.185662 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3d61e-b7c5-414a-b03f-3decde43d3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:13.186003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3d61e-b7c5-414a-b03f-3decde43d3d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:13.186727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3d61e-b7c5-414a-b03f-3decde43d3d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:13.188491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3d61e-b7c5-414a-b03f-3decde43d3d7 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:13.203839 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1179/2366] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:25:13.204980 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870629c-d1af-45b4-84b2-dc8e4e8e5535 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:13.217778 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870629c-d1af-45b4-84b2-dc8e4e8e5535 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:13.223159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431ae09-9ebd-4316-85fd-37439f524568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:13.223897 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431ae09-9ebd-4316-85fd-37439f524568 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:13.224398 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431ae09-9ebd-4316-85fd-37439f524568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:13.224756 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431ae09-9ebd-4316-85fd-37439f524568 admin admin] PATH_INFO: `/v3/projects/23960fb910914891918c413fd9221dba/users/8cf384c8aeb342ca8ce5de0cd29f04cb/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:13.243664 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e431ae09-9ebd-4316-85fd-37439f524568 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23960fb910914891918c413fd9221dba, user_id=8cf384c8aeb342ca8ce5de0cd29f04cb, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:13.246937 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e431ae09-9ebd-4316-85fd-37439f524568 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:13.265885 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1188/2367] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:25:13 2022] PUT /identity/v3/projects/23960fb910914891918c413fd9221dba/users/8cf384c8aeb342ca8ce5de0cd29f04cb/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:25:13.279136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81e8cb-772e-4275-b59a-135a704b4e80 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:13.279136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81e8cb-772e-4275-b59a-135a704b4e80 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:13.279136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81e8cb-772e-4275-b59a-135a704b4e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:13.332059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-8c81e8cb-772e-4275-b59a-135a704b4e80 None None] MFA Rules not processed for user `8cf384c8aeb342ca8ce5de0cd29f04cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:25:13.333948 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-8c81e8cb-772e-4275-b59a-135a704b4e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:13.443537 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1180/2368] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:25:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:25:13.444860 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431ae09-9ebd-4316-85fd-37439f524568 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:13.513090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431ae09-9ebd-4316-85fd-37439f524568 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:13.517847 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074de0b-5123-46b5-b3c9-021a936adebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:13.518460 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074de0b-5123-46b5-b3c9-021a936adebf admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:13.518763 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074de0b-5123-46b5-b3c9-021a936adebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:13.519015 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074de0b-5123-46b5-b3c9-021a936adebf admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:13.519678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2074de0b-5123-46b5-b3c9-021a936adebf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:13.522747 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2074de0b-5123-46b5-b3c9-021a936adebf admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:13.551453 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1189/2369] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:25:13 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:25:13.556017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81e8cb-772e-4275-b59a-135a704b4e80 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:13.566865 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81e8cb-772e-4275-b59a-135a704b4e80 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:13.625149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0f0cf-1a48-46e0-b71e-da828fcacb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:13.625901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0f0cf-1a48-46e0-b71e-da828fcacb84 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:13.626380 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0f0cf-1a48-46e0-b71e-da828fcacb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:13.626685 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0f0cf-1a48-46e0-b71e-da828fcacb84 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:13.627467 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f0f0cf-1a48-46e0-b71e-da828fcacb84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:13.630368 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f0f0cf-1a48-46e0-b71e-da828fcacb84 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:13.652155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1181/2370] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:25:13 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:25:13.656078 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074de0b-5123-46b5-b3c9-021a936adebf admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:13.667610 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074de0b-5123-46b5-b3c9-021a936adebf admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:13.672544 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8117312-1b8a-4eca-b15b-6bca45035bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:13.673147 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8117312-1b8a-4eca-b15b-6bca45035bbc admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:13.673434 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8117312-1b8a-4eca-b15b-6bca45035bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:13.673730 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8117312-1b8a-4eca-b15b-6bca45035bbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:13.674405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8117312-1b8a-4eca-b15b-6bca45035bbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:13.676052 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8117312-1b8a-4eca-b15b-6bca45035bbc admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:13.687343 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1190/2371] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:25:13.691506 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0f0cf-1a48-46e0-b71e-da828fcacb84 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:13.701929 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0f0cf-1a48-46e0-b71e-da828fcacb84 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:13.706512 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56904ea3-3f46-4fdc-97c0-77ec831176fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:13.707242 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56904ea3-3f46-4fdc-97c0-77ec831176fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:13.707521 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56904ea3-3f46-4fdc-97c0-77ec831176fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:13.707790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56904ea3-3f46-4fdc-97c0-77ec831176fd admin admin] PATH_INFO: `/v3/projects/1f7375b7f1d44014b9238f6b5c71f309/users/20b2039e49b6470fac1a0fea1fdb33df/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:13.722635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56904ea3-3f46-4fdc-97c0-77ec831176fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f7375b7f1d44014b9238f6b5c71f309, user_id=20b2039e49b6470fac1a0fea1fdb33df, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:13.725329 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56904ea3-3f46-4fdc-97c0-77ec831176fd admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:13.727545 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:25:13.743448 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1182/2372] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:25:13 2022] PUT /identity/v3/projects/1f7375b7f1d44014b9238f6b5c71f309/users/20b2039e49b6470fac1a0fea1fdb33df/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:25:13.747537 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8117312-1b8a-4eca-b15b-6bca45035bbc admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:13.758580 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8117312-1b8a-4eca-b15b-6bca45035bbc admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:13.814238 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2c2a2-3422-40d0-98c7-1cbfffa86d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:13.814847 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2c2a2-3422-40d0-98c7-1cbfffa86d17 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:13.815161 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2c2a2-3422-40d0-98c7-1cbfffa86d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:13.815440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2c2a2-3422-40d0-98c7-1cbfffa86d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:13.816084 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d2c2a2-3422-40d0-98c7-1cbfffa86d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:13.817734 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d2c2a2-3422-40d0-98c7-1cbfffa86d17 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:13.828302 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1191/2373] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:25:13.832622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56904ea3-3f46-4fdc-97c0-77ec831176fd admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:13.843990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56904ea3-3f46-4fdc-97c0-77ec831176fd admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:13.848643 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec6e7a-b49a-4345-9d83-e205edb5d4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:13.849642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec6e7a-b49a-4345-9d83-e205edb5d4ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:13.850033 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec6e7a-b49a-4345-9d83-e205edb5d4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:13.850319 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec6e7a-b49a-4345-9d83-e205edb5d4ad admin admin] PATH_INFO: `/v3/projects/1f7375b7f1d44014b9238f6b5c71f309/users/20b2039e49b6470fac1a0fea1fdb33df/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:13.852278 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eec6e7a-b49a-4345-9d83-e205edb5d4ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f7375b7f1d44014b9238f6b5c71f309, user_id=20b2039e49b6470fac1a0fea1fdb33df, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:13.854906 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eec6e7a-b49a-4345-9d83-e205edb5d4ad admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:13.865854 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1183/2374] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:25:13 2022] PUT /identity/v3/projects/1f7375b7f1d44014b9238f6b5c71f309/users/20b2039e49b6470fac1a0fea1fdb33df/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:25:13.877165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a59e5-c08e-4f30-a621-5157e5f760a8 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:13.877451 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a59e5-c08e-4f30-a621-5157e5f760a8 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:13.877725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a59e5-c08e-4f30-a621-5157e5f760a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:13.906040 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-052a59e5-c08e-4f30-a621-5157e5f760a8 None None] MFA Rules not processed for user `20b2039e49b6470fac1a0fea1fdb33df`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:25:13.907282 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-052a59e5-c08e-4f30-a621-5157e5f760a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:13.915426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:25:13.978397 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1192/2375] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:25:13 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:25:13.989441 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec6e7a-b49a-4345-9d83-e205edb5d4ad admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:13.991661 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-1eec6e7a-b49a-4345-9d83-e205edb5d4ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:14.049909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec6e7a-b49a-4345-9d83-e205edb5d4ad admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:14.054238 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c034c4-5cb1-4fd2-aa46-3c44f57ad5d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:14.054838 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c034c4-5cb1-4fd2-aa46-3c44f57ad5d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:14.055142 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c034c4-5cb1-4fd2-aa46-3c44f57ad5d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:14.055417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c034c4-5cb1-4fd2-aa46-3c44f57ad5d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:14.116956 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c034c4-5cb1-4fd2-aa46-3c44f57ad5d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:14.118799 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c034c4-5cb1-4fd2-aa46-3c44f57ad5d6 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:14.127953 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1184/2376] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:25:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:25:16.750552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a59e5-c08e-4f30-a621-5157e5f760a8 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:16.762917 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a59e5-c08e-4f30-a621-5157e5f760a8 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:16.772316 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b14107-a3ae-4d1a-9229-d6954f44601f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:16.772945 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b14107-a3ae-4d1a-9229-d6954f44601f service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:16.773216 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b14107-a3ae-4d1a-9229-d6954f44601f service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:16.773488 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b14107-a3ae-4d1a-9229-d6954f44601f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:16.869881 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b14107-a3ae-4d1a-9229-d6954f44601f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:16.872696 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b14107-a3ae-4d1a-9229-d6954f44601f service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:16.886424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1193/2377] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:25:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:25:17.450114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c034c4-5cb1-4fd2-aa46-3c44f57ad5d6 service cinder] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:17.459673 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c034c4-5cb1-4fd2-aa46-3c44f57ad5d6 service cinder] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:17.463837 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca09855-f180-44ee-ba55-4efd70a45490 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:17.464470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca09855-f180-44ee-ba55-4efd70a45490 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:17.464774 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca09855-f180-44ee-ba55-4efd70a45490 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:17.465029 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca09855-f180-44ee-ba55-4efd70a45490 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:17.476136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca09855-f180-44ee-ba55-4efd70a45490 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:17.477793 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca09855-f180-44ee-ba55-4efd70a45490 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:17.488727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1185/2378] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:25:17 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:25:22.434176 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b14107-a3ae-4d1a-9229-d6954f44601f service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:22.498828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b14107-a3ae-4d1a-9229-d6954f44601f service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:22.503524 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f089d2-8569-4165-b5c2-453f6266fa7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:22.504154 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f089d2-8569-4165-b5c2-453f6266fa7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:22.504435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f089d2-8569-4165-b5c2-453f6266fa7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:22.504737 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f089d2-8569-4165-b5c2-453f6266fa7b admin admin] PATH_INFO: `/v3/users/ea546db2fc8748c990fe65ff00e493a7` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:22.506511 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f089d2-8569-4165-b5c2-453f6266fa7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea546db2fc8748c990fe65ff00e493a7)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:22.508498 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f089d2-8569-4165-b5c2-453f6266fa7b admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:22.553443 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:25:22.555636 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-31f089d2-8569-4165-b5c2-453f6266fa7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea546db2fc8748c990fe65ff00e493a7'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:22.564150 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-31f089d2-8569-4165-b5c2-453f6266fa7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea546db2fc8748c990fe65ff00e493a7'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:22.572524 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-31f089d2-8569-4165-b5c2-453f6266fa7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea546db2fc8748c990fe65ff00e493a7'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:22.573163 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-31f089d2-8569-4165-b5c2-453f6266fa7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea546db2fc8748c990fe65ff00e493a7'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:22.573769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-31f089d2-8569-4165-b5c2-453f6266fa7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea546db2fc8748c990fe65ff00e493a7'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:22.591487 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1194/2379] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:25:22 2022] DELETE /identity/v3/users/ea546db2fc8748c990fe65ff00e493a7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:25:22.595901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d37a2e-3589-43f4-a74c-df5ba23f6a4e None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:22.596364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d37a2e-3589-43f4-a74c-df5ba23f6a4e None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:22.596457 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d37a2e-3589-43f4-a74c-df5ba23f6a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:22.617713 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-f2d37a2e-3589-43f4-a74c-df5ba23f6a4e None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:25:22.618975 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f2d37a2e-3589-43f4-a74c-df5ba23f6a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:22.678202 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1186/2380] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:25:22.688274 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f089d2-8569-4165-b5c2-453f6266fa7b admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:22.689714 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-31f089d2-8569-4165-b5c2-453f6266fa7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:22.747477 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f089d2-8569-4165-b5c2-453f6266fa7b admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:22.751139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1576d6-8bc4-4966-aae4-b13c11a686d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:22.751742 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1576d6-8bc4-4966-aae4-b13c11a686d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:22.752022 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1576d6-8bc4-4966-aae4-b13c11a686d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:22.752295 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1576d6-8bc4-4966-aae4-b13c11a686d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:22.761608 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1576d6-8bc4-4966-aae4-b13c11a686d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:22.763554 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1576d6-8bc4-4966-aae4-b13c11a686d5 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:22.774483 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1195/2381] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:25:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:25:23.192501 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d37a2e-3589-43f4-a74c-df5ba23f6a4e None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:23.193932 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f2d37a2e-3589-43f4-a74c-df5ba23f6a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:23.206123 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d37a2e-3589-43f4-a74c-df5ba23f6a4e None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:23.211443 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce061f3-bb63-44d1-9cef-7698a1cc378b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:23.212149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce061f3-bb63-44d1-9cef-7698a1cc378b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:23.212509 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce061f3-bb63-44d1-9cef-7698a1cc378b admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:23.212847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce061f3-bb63-44d1-9cef-7698a1cc378b admin admin] PATH_INFO: `/v3/projects/05abcf8cdc5a4def8ac0d1a5720b1f6b` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:23.213962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce061f3-bb63-44d1-9cef-7698a1cc378b admin admin] RBAC: Authorizing `identity:delete_project(project_id=05abcf8cdc5a4def8ac0d1a5720b1f6b)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:23.215905 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce061f3-bb63-44d1-9cef-7698a1cc378b admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:23.246403 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:25:23.284221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-2ce061f3-bb63-44d1-9cef-7698a1cc378b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05abcf8cdc5a4def8ac0d1a5720b1f6b', 'request_id': 'req-2ce061f3-bb63-44d1-9cef-7698a1cc378b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:23.298427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-2ce061f3-bb63-44d1-9cef-7698a1cc378b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05abcf8cdc5a4def8ac0d1a5720b1f6b', 'request_id': 'req-2ce061f3-bb63-44d1-9cef-7698a1cc378b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:23.304392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-2ce061f3-bb63-44d1-9cef-7698a1cc378b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05abcf8cdc5a4def8ac0d1a5720b1f6b', 'request_id': 'req-2ce061f3-bb63-44d1-9cef-7698a1cc378b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:23.317026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-2ce061f3-bb63-44d1-9cef-7698a1cc378b admin admin] The token cache is being invalidate because project 05abcf8cdc5a4def8ac0d1a5720b1f6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:25:23.317404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-2ce061f3-bb63-44d1-9cef-7698a1cc378b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:23.318174 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-2ce061f3-bb63-44d1-9cef-7698a1cc378b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:23.320114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1187/2382] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:25:23 2022] DELETE /identity/v3/projects/05abcf8cdc5a4def8ac0d1a5720b1f6b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:25:23.324017 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1576d6-8bc4-4966-aae4-b13c11a686d5 service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:23.335160 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1576d6-8bc4-4966-aae4-b13c11a686d5 service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:23.391994 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a18b8-7074-47c7-a3cc-36a4d018dc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:23.392532 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a18b8-7074-47c7-a3cc-36a4d018dc4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:23.392807 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a18b8-7074-47c7-a3cc-36a4d018dc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:23.393078 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a18b8-7074-47c7-a3cc-36a4d018dc4b admin admin] PATH_INFO: `/v3/users/8cf384c8aeb342ca8ce5de0cd29f04cb` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:23.394195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a18b8-7074-47c7-a3cc-36a4d018dc4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cf384c8aeb342ca8ce5de0cd29f04cb)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:23.396142 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a18b8-7074-47c7-a3cc-36a4d018dc4b admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:23.442963 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-473a18b8-7074-47c7-a3cc-36a4d018dc4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cf384c8aeb342ca8ce5de0cd29f04cb'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:23.451022 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-473a18b8-7074-47c7-a3cc-36a4d018dc4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cf384c8aeb342ca8ce5de0cd29f04cb'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:23.457908 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-473a18b8-7074-47c7-a3cc-36a4d018dc4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cf384c8aeb342ca8ce5de0cd29f04cb'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:23.458535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-473a18b8-7074-47c7-a3cc-36a4d018dc4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cf384c8aeb342ca8ce5de0cd29f04cb'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:23.460008 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-473a18b8-7074-47c7-a3cc-36a4d018dc4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cf384c8aeb342ca8ce5de0cd29f04cb'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:23.464467 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:25:23.479302 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1196/2383] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:25:23 2022] DELETE /identity/v3/users/8cf384c8aeb342ca8ce5de0cd29f04cb => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:25:23.764423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce061f3-bb63-44d1-9cef-7698a1cc378b admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:23.766443 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-2ce061f3-bb63-44d1-9cef-7698a1cc378b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:23.789017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce061f3-bb63-44d1-9cef-7698a1cc378b admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:23.869105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeacb52-7df2-40c6-9247-5a0330cf0bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:23.869767 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeacb52-7df2-40c6-9247-5a0330cf0bcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:23.870052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeacb52-7df2-40c6-9247-5a0330cf0bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:23.870327 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeacb52-7df2-40c6-9247-5a0330cf0bcb admin admin] PATH_INFO: `/v3/projects/23960fb910914891918c413fd9221dba` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:23.871514 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeacb52-7df2-40c6-9247-5a0330cf0bcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=23960fb910914891918c413fd9221dba)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:23.873591 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeacb52-7df2-40c6-9247-5a0330cf0bcb admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:23.944635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-0eeacb52-7df2-40c6-9247-5a0330cf0bcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23960fb910914891918c413fd9221dba', 'request_id': 'req-0eeacb52-7df2-40c6-9247-5a0330cf0bcb'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:23.953920 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-0eeacb52-7df2-40c6-9247-5a0330cf0bcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23960fb910914891918c413fd9221dba', 'request_id': 'req-0eeacb52-7df2-40c6-9247-5a0330cf0bcb'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:23.963376 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-0eeacb52-7df2-40c6-9247-5a0330cf0bcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23960fb910914891918c413fd9221dba', 'request_id': 'req-0eeacb52-7df2-40c6-9247-5a0330cf0bcb'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:23.978078 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-0eeacb52-7df2-40c6-9247-5a0330cf0bcb admin admin] The token cache is being invalidate because project 23960fb910914891918c413fd9221dba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:25:23.978473 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-0eeacb52-7df2-40c6-9247-5a0330cf0bcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:23.979079 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-0eeacb52-7df2-40c6-9247-5a0330cf0bcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:23.980753 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1188/2384] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:25:23 2022] DELETE /identity/v3/projects/23960fb910914891918c413fd9221dba => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:25:23.984815 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a18b8-7074-47c7-a3cc-36a4d018dc4b admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:23.986351 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-473a18b8-7074-47c7-a3cc-36a4d018dc4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:24.003346 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a18b8-7074-47c7-a3cc-36a4d018dc4b admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:24.023723 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f541290-0513-453e-93e5-21639fa1705c None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:24.024239 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f541290-0513-453e-93e5-21639fa1705c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:24.024585 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f541290-0513-453e-93e5-21639fa1705c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:24.048596 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-2f541290-0513-453e-93e5-21639fa1705c None None] MFA Rules not processed for user `a8fb7a9acefd4b418c992661037c2ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:25:24.050301 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-2f541290-0513-453e-93e5-21639fa1705c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:24.072805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6266d1-c9ab-4754-ad10-d64ed955a03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:24.073486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6266d1-c9ab-4754-ad10-d64ed955a03e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:24.074410 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6266d1-c9ab-4754-ad10-d64ed955a03e admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:24.074700 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6266d1-c9ab-4754-ad10-d64ed955a03e admin admin] PATH_INFO: `/v3/users/20b2039e49b6470fac1a0fea1fdb33df` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:24.077151 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6266d1-c9ab-4754-ad10-d64ed955a03e admin admin] RBAC: Authorizing `identity:delete_user(user_id=20b2039e49b6470fac1a0fea1fdb33df)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:24.080309 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6266d1-c9ab-4754-ad10-d64ed955a03e admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:24.088259 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:25:24.125696 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1189/2385] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 31 00:25:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:25:24.141997 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3c6266d1-c9ab-4754-ad10-d64ed955a03e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20b2039e49b6470fac1a0fea1fdb33df'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:24.150520 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3c6266d1-c9ab-4754-ad10-d64ed955a03e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20b2039e49b6470fac1a0fea1fdb33df'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:24.163479 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3c6266d1-c9ab-4754-ad10-d64ed955a03e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20b2039e49b6470fac1a0fea1fdb33df'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:24.164590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3c6266d1-c9ab-4754-ad10-d64ed955a03e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20b2039e49b6470fac1a0fea1fdb33df'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:24.165333 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3c6266d1-c9ab-4754-ad10-d64ed955a03e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20b2039e49b6470fac1a0fea1fdb33df'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:24.193726 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f541290-0513-453e-93e5-21639fa1705c None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:24.195981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-2f541290-0513-453e-93e5-21639fa1705c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:24.197982 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1197/2386] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:25:23 2022] DELETE /identity/v3/users/20b2039e49b6470fac1a0fea1fdb33df => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:25:24.282166 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f541290-0513-453e-93e5-21639fa1705c None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:24.366595 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4851ca-7564-4973-a50c-0ec0cdf8f229 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd3cfbc8ebd4c5a83bef09efe10bc91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:24.367267 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4851ca-7564-4973-a50c-0ec0cdf8f229 service swift] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:24.367547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4851ca-7564-4973-a50c-0ec0cdf8f229 service swift] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:24.367951 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4851ca-7564-4973-a50c-0ec0cdf8f229 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:24.369785 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-8b4851ca-7564-4973-a50c-0ec0cdf8f229 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:24.437232 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4851ca-7564-4973-a50c-0ec0cdf8f229 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:24.439131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4851ca-7564-4973-a50c-0ec0cdf8f229 service swift] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:24.450717 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1190/2387] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 31 00:25:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:25:24.621331 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6266d1-c9ab-4754-ad10-d64ed955a03e admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:24.624391 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-3c6266d1-c9ab-4754-ad10-d64ed955a03e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:24.742468 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6266d1-c9ab-4754-ad10-d64ed955a03e admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:24.748887 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5301d5a-c488-409e-91ee-d17144377e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:24.749770 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5301d5a-c488-409e-91ee-d17144377e62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:24.750214 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5301d5a-c488-409e-91ee-d17144377e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:24.750648 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5301d5a-c488-409e-91ee-d17144377e62 admin admin] PATH_INFO: `/v3/projects/1f7375b7f1d44014b9238f6b5c71f309` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:24.754768 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5301d5a-c488-409e-91ee-d17144377e62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f7375b7f1d44014b9238f6b5c71f309)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:24.757055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5301d5a-c488-409e-91ee-d17144377e62 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:24.766818 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:25:24.813925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-f5301d5a-c488-409e-91ee-d17144377e62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f7375b7f1d44014b9238f6b5c71f309', 'request_id': 'req-f5301d5a-c488-409e-91ee-d17144377e62'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:24.820218 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-f5301d5a-c488-409e-91ee-d17144377e62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f7375b7f1d44014b9238f6b5c71f309', 'request_id': 'req-f5301d5a-c488-409e-91ee-d17144377e62'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:24.826977 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-f5301d5a-c488-409e-91ee-d17144377e62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f7375b7f1d44014b9238f6b5c71f309', 'request_id': 'req-f5301d5a-c488-409e-91ee-d17144377e62'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:24.842147 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-f5301d5a-c488-409e-91ee-d17144377e62 admin admin] The token cache is being invalidate because project 1f7375b7f1d44014b9238f6b5c71f309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:25:24.842527 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-f5301d5a-c488-409e-91ee-d17144377e62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:24.843272 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-f5301d5a-c488-409e-91ee-d17144377e62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:25:24.846788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1198/2388] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:25:24 2022] DELETE /identity/v3/projects/1f7375b7f1d44014b9238f6b5c71f309 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:25:29.557263 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4851ca-7564-4973-a50c-0ec0cdf8f229 service swift] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:29.558847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-8b4851ca-7564-4973-a50c-0ec0cdf8f229 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:29.590712 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5301d5a-c488-409e-91ee-d17144377e62 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:29.596228 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f5301d5a-c488-409e-91ee-d17144377e62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:29.633411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:25:29.682502 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5301d5a-c488-409e-91ee-d17144377e62 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:29.688962 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73479f8-e93c-4ad3-a553-b353fb8b44ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:29.689664 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73479f8-e93c-4ad3-a553-b353fb8b44ab service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:29.689991 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73479f8-e93c-4ad3-a553-b353fb8b44ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:29.690489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73479f8-e93c-4ad3-a553-b353fb8b44ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:29.693224 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f73479f8-e93c-4ad3-a553-b353fb8b44ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:29.699717 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4851ca-7564-4973-a50c-0ec0cdf8f229 service swift] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:29.703606 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778a66b-6490-4684-b87b-53846c1d9b4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:29.704948 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778a66b-6490-4684-b87b-53846c1d9b4b service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:29.704948 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778a66b-6490-4684-b87b-53846c1d9b4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:29.705218 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778a66b-6490-4684-b87b-53846c1d9b4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:29.707761 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-0778a66b-6490-4684-b87b-53846c1d9b4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:29.813300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73479f8-e93c-4ad3-a553-b353fb8b44ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:29.816125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73479f8-e93c-4ad3-a553-b353fb8b44ab service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:29.820802 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778a66b-6490-4684-b87b-53846c1d9b4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:29.822313 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778a66b-6490-4684-b87b-53846c1d9b4b service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:29.836036 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1199/2389] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:25:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:25:29.836589 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1191/2390] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:25:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:25:39.829996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=43 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:25:39.833348 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:25:43.674308 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73479f8-e93c-4ad3-a553-b353fb8b44ab service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:43.676380 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f73479f8-e93c-4ad3-a553-b353fb8b44ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:43.695904 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73479f8-e93c-4ad3-a553-b353fb8b44ab service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:43.705613 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d193d-e3c4-4334-8336-a38428c9e0c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:43.706241 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d193d-e3c4-4334-8336-a38428c9e0c6 service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:43.706557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d193d-e3c4-4334-8336-a38428c9e0c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:43.706799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d193d-e3c4-4334-8336-a38428c9e0c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:43.708609 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-bc9d193d-e3c4-4334-8336-a38428c9e0c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:43.771430 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d193d-e3c4-4334-8336-a38428c9e0c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:43.774297 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d193d-e3c4-4334-8336-a38428c9e0c6 service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:43.784049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1200/2391] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:25:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:25:47.192512 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778a66b-6490-4684-b87b-53846c1d9b4b service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:47.201683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778a66b-6490-4684-b87b-53846c1d9b4b service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:47.205378 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2f491-dd96-4c66-8bfd-a7a43a667b29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:47.206016 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2f491-dd96-4c66-8bfd-a7a43a667b29 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:47.206295 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2f491-dd96-4c66-8bfd-a7a43a667b29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:47.206567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2f491-dd96-4c66-8bfd-a7a43a667b29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:47.208307 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-7ac2f491-dd96-4c66-8bfd-a7a43a667b29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:47.218463 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac2f491-dd96-4c66-8bfd-a7a43a667b29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:47.219901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac2f491-dd96-4c66-8bfd-a7a43a667b29 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:47.228874 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1192/2392] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:25:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:25:47.352058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1151a4d-8ed7-4c3c-bda1-e46b8ffecede None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:47.352652 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1151a4d-8ed7-4c3c-bda1-e46b8ffecede None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:47.352652 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1151a4d-8ed7-4c3c-bda1-e46b8ffecede None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:47.354012 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1201/2393] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 31 00:25:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:25:47.360504 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8dca9-710d-4cc6-b6d9-f30de9761b43 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:47.360790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8dca9-710d-4cc6-b6d9-f30de9761b43 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:47.361062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8dca9-710d-4cc6-b6d9-f30de9761b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:47.369124 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1151a4d-8ed7-4c3c-bda1-e46b8ffecede None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:47.380399 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1151a4d-8ed7-4c3c-bda1-e46b8ffecede None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:47.384389 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-c6f8dca9-710d-4cc6-b6d9-f30de9761b43 None None] MFA Rules not processed for user `3aa53ae5a30146f69d7d6eefe4290dba`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:25:47.384736 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48701dcc-4c55-46a5-ac7b-d79ac1b074a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:47.385341 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48701dcc-4c55-46a5-ac7b-d79ac1b074a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:47.385606 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c6f8dca9-710d-4cc6-b6d9-f30de9761b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:47.385718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48701dcc-4c55-46a5-ac7b-d79ac1b074a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:47.385876 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48701dcc-4c55-46a5-ac7b-d79ac1b074a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:47.387941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-48701dcc-4c55-46a5-ac7b-d79ac1b074a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:47.395357 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1193/2394] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 31 00:25:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:25:47.408221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8dca9-710d-4cc6-b6d9-f30de9761b43 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:47.409651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c6f8dca9-710d-4cc6-b6d9-f30de9761b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:47.453416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48701dcc-4c55-46a5-ac7b-d79ac1b074a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:47.454771 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48701dcc-4c55-46a5-ac7b-d79ac1b074a2 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:47.465207 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1202/2395] 10.222.0.67 () {60 vars in 1396 bytes} [Wed Aug 31 00:25:47 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3266 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:25:47.471299 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8dca9-710d-4cc6-b6d9-f30de9761b43 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:47.474909 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b199bed7-b50d-407e-92f6-5cf6063f32b6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55714a21c1cb4f2b949603230243ed02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:47.475646 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199bed7-b50d-407e-92f6-5cf6063f32b6 service glance] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:47.475934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199bed7-b50d-407e-92f6-5cf6063f32b6 service glance] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:47.476202 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199bed7-b50d-407e-92f6-5cf6063f32b6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:47.487993 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b199bed7-b50d-407e-92f6-5cf6063f32b6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:47.489422 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b199bed7-b50d-407e-92f6-5cf6063f32b6 service glance] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:47.500256 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1194/2396] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:25:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:25:47.576179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165903b-034d-4d8f-9b7b-f6f5c281a3c4 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:47.576472 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165903b-034d-4d8f-9b7b-f6f5c281a3c4 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:47.576753 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165903b-034d-4d8f-9b7b-f6f5c281a3c4 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:47.578123 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1203/2397] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 31 00:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:25:47.585674 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c90f62-d0ed-4114-a07c-0a310a82526c None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:47.585674 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c90f62-d0ed-4114-a07c-0a310a82526c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:47.585674 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c90f62-d0ed-4114-a07c-0a310a82526c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:47.604500 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-d8c90f62-d0ed-4114-a07c-0a310a82526c None None] MFA Rules not processed for user `05c0d9554cf94c41a2cd709cab378e53`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:25:47.605720 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-d8c90f62-d0ed-4114-a07c-0a310a82526c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:47.612333 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1195/2398] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 31 00:25:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:25:47.624243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2165903b-034d-4d8f-9b7b-f6f5c281a3c4 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:47.625931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-2165903b-034d-4d8f-9b7b-f6f5c281a3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:47.640174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2165903b-034d-4d8f-9b7b-f6f5c281a3c4 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:47.643789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a8d91-4619-4c53-a4d7-977676aa1915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:47.644391 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a8d91-4619-4c53-a4d7-977676aa1915 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:47.644672 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a8d91-4619-4c53-a4d7-977676aa1915 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:47.644945 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a8d91-4619-4c53-a4d7-977676aa1915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:47.652787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a8d91-4619-4c53-a4d7-977676aa1915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:47.654208 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a8d91-4619-4c53-a4d7-977676aa1915 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:47.661998 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1204/2399] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:25:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:25:47.751678 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300f6b3-db70-4e8b-ba1f-21bdb118b113 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:47.751967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300f6b3-db70-4e8b-ba1f-21bdb118b113 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:47.752245 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300f6b3-db70-4e8b-ba1f-21bdb118b113 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:47.753742 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1196/2400] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 31 00:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:25:47.759961 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a407d-6621-412a-a55f-84c2465b0f22 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:47.760246 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a407d-6621-412a-a55f-84c2465b0f22 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:47.760518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a407d-6621-412a-a55f-84c2465b0f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:47.781628 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-4b3a407d-6621-412a-a55f-84c2465b0f22 None None] MFA Rules not processed for user `3aa53ae5a30146f69d7d6eefe4290dba`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:25:47.782953 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4b3a407d-6621-412a-a55f-84c2465b0f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:47.794434 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1205/2401] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 31 00:25:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:25:47.806287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300f6b3-db70-4e8b-ba1f-21bdb118b113 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:47.818687 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300f6b3-db70-4e8b-ba1f-21bdb118b113 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:47.823572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612223a9-fe04-45a8-94c8-03b7604cb63e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:47.824182 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612223a9-fe04-45a8-94c8-03b7604cb63e service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:47.824459 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612223a9-fe04-45a8-94c8-03b7604cb63e service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:47.824727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612223a9-fe04-45a8-94c8-03b7604cb63e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:47.837226 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612223a9-fe04-45a8-94c8-03b7604cb63e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:47.838688 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612223a9-fe04-45a8-94c8-03b7604cb63e service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:47.850435 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1197/2402] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:25:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:25:47.915728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a407d-6621-412a-a55f-84c2465b0f22 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:47.917155 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4b3a407d-6621-412a-a55f-84c2465b0f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:47.996716 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a407d-6621-412a-a55f-84c2465b0f22 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:48.001858 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f727b7c-efce-4531-bd99-5241015e0e33 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdacdcb2451742938164fdd44cdbaebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:48.002467 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f727b7c-efce-4531-bd99-5241015e0e33 service placement] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:48.002752 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f727b7c-efce-4531-bd99-5241015e0e33 service placement] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:48.003024 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f727b7c-efce-4531-bd99-5241015e0e33 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:48.004837 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-2f727b7c-efce-4531-bd99-5241015e0e33 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:48.017286 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f727b7c-efce-4531-bd99-5241015e0e33 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:48.018872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f727b7c-efce-4531-bd99-5241015e0e33 service placement] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:48.029051 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1206/2403] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:25:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:25:52.767785 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99aa089-b348-4304-869d-4f5272809221 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:52.768254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99aa089-b348-4304-869d-4f5272809221 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:52.768537 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99aa089-b348-4304-869d-4f5272809221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:52.793739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-b99aa089-b348-4304-869d-4f5272809221 None None] MFA Rules not processed for user `a8fb7a9acefd4b418c992661037c2ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:25:52.794975 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b99aa089-b348-4304-869d-4f5272809221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:52.853451 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1198/2404] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 31 00:25:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:25:52.865642 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f727b7c-efce-4531-bd99-5241015e0e33 service placement] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:25:52.867265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-2f727b7c-efce-4531-bd99-5241015e0e33 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:25:52.876752 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:25:52.942935 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f727b7c-efce-4531-bd99-5241015e0e33 service placement] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:25:52.948746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb5566-9c8b-4c22-9548-f8641ea917fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd3cfbc8ebd4c5a83bef09efe10bc91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:25:52.949355 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb5566-9c8b-4c22-9548-f8641ea917fb service swift] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:25:52.949634 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb5566-9c8b-4c22-9548-f8641ea917fb service swift] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:25:52.949904 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb5566-9c8b-4c22-9548-f8641ea917fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:25:52.957979 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb5566-9c8b-4c22-9548-f8641ea917fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:25:52.959614 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb5566-9c8b-4c22-9548-f8641ea917fb service swift] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:25:52.968319 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1207/2405] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 31 00:25:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:26:02.846509 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=69 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:02.964887 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=29 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:23.703497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09be9b-27f6-4ef2-bff0-ccc9f94dabfd None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:23.704104 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09be9b-27f6-4ef2-bff0-ccc9f94dabfd None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:23.704104 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09be9b-27f6-4ef2-bff0-ccc9f94dabfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:23.723912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-ed09be9b-27f6-4ef2-bff0-ccc9f94dabfd None None] MFA Rules not processed for user `b03932102b7541f89a60a9c535a42e76`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:26:23.725126 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-ed09be9b-27f6-4ef2-bff0-ccc9f94dabfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:23.782940 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1199/2406] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:26:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:26:24.014411 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb5566-9c8b-4c22-9548-f8641ea917fb service swift] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:26:24.024047 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb5566-9c8b-4c22-9548-f8641ea917fb service swift] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:26:24.028432 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693cf89-555f-4228-8c29-902eb4627695 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:26:24.029023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693cf89-555f-4228-8c29-902eb4627695 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:24.029299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693cf89-555f-4228-8c29-902eb4627695 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:24.029566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693cf89-555f-4228-8c29-902eb4627695 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:24.031304 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-6693cf89-555f-4228-8c29-902eb4627695 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:24.041256 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6693cf89-555f-4228-8c29-902eb4627695 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:26:24.043084 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6693cf89-555f-4228-8c29-902eb4627695 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:26:24.056391 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1208/2407] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:26:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:26:25.012082 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09be9b-27f6-4ef2-bff0-ccc9f94dabfd None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:26:25.030022 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09be9b-27f6-4ef2-bff0-ccc9f94dabfd None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:26:25.036892 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011d5a3-2139-41d0-8bcf-95a00a4c9b6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:26:25.037520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011d5a3-2139-41d0-8bcf-95a00a4c9b6c service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:25.037813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011d5a3-2139-41d0-8bcf-95a00a4c9b6c service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:25.038089 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011d5a3-2139-41d0-8bcf-95a00a4c9b6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:25.040068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-4011d5a3-2139-41d0-8bcf-95a00a4c9b6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:25.057450 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4011d5a3-2139-41d0-8bcf-95a00a4c9b6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:26:25.059241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4011d5a3-2139-41d0-8bcf-95a00a4c9b6c service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:26:25.070826 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1200/2408] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:26:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:26:26.018192 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693cf89-555f-4228-8c29-902eb4627695 service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:26:26.028522 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693cf89-555f-4228-8c29-902eb4627695 service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:26:26.032608 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff30a88-7432-4edc-965b-52ac5d0433b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:26:26.033245 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff30a88-7432-4edc-965b-52ac5d0433b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:26.033522 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff30a88-7432-4edc-965b-52ac5d0433b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:26.033794 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff30a88-7432-4edc-965b-52ac5d0433b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:26.035562 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-aff30a88-7432-4edc-965b-52ac5d0433b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:26.046131 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff30a88-7432-4edc-965b-52ac5d0433b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:26:26.047651 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff30a88-7432-4edc-965b-52ac5d0433b6 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:26:26.056156 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1209/2409] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:26:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:26:28.680913 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011d5a3-2139-41d0-8bcf-95a00a4c9b6c service nova] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:26:28.683773 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-4011d5a3-2139-41d0-8bcf-95a00a4c9b6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:28.700238 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011d5a3-2139-41d0-8bcf-95a00a4c9b6c service nova] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:26:28.706680 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c038ef-fd42-457d-b418-c013761479d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:26:28.707320 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c038ef-fd42-457d-b418-c013761479d6 service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:28.707596 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c038ef-fd42-457d-b418-c013761479d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:28.707870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c038ef-fd42-457d-b418-c013761479d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:28.709588 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f6c038ef-fd42-457d-b418-c013761479d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:28.722048 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c038ef-fd42-457d-b418-c013761479d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:26:28.723744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c038ef-fd42-457d-b418-c013761479d6 service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:26:28.735104 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1201/2410] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:26:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:26:28.875651 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3061eba-4b2b-40f2-bb0d-efaa71667945 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:28.876041 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3061eba-4b2b-40f2-bb0d-efaa71667945 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:28.876194 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3061eba-4b2b-40f2-bb0d-efaa71667945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:28.896284 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-d3061eba-4b2b-40f2-bb0d-efaa71667945 None None] MFA Rules not processed for user `a8fb7a9acefd4b418c992661037c2ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:26:28.897865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-d3061eba-4b2b-40f2-bb0d-efaa71667945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:28.911184 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1210/2411] 10.222.0.67 () {58 vars in 967 bytes} [Wed Aug 31 00:26:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:26:28.934839 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c038ef-fd42-457d-b418-c013761479d6 service nova] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:26:28.946065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c038ef-fd42-457d-b418-c013761479d6 service nova] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:26:28.951621 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9971a4-08f7-439a-95aa-64445dca3850 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd3cfbc8ebd4c5a83bef09efe10bc91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:26:28.952164 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9971a4-08f7-439a-95aa-64445dca3850 service swift] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:28.952445 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9971a4-08f7-439a-95aa-64445dca3850 service swift] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:28.952726 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9971a4-08f7-439a-95aa-64445dca3850 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:28.961603 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9971a4-08f7-439a-95aa-64445dca3850 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:26:28.963902 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9971a4-08f7-439a-95aa-64445dca3850 service swift] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:26:28.974550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1202/2412] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 31 00:26:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:26:38.890281 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=15 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:38.969116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=43 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:26:51.314627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3061eba-4b2b-40f2-bb0d-efaa71667945 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:26:51.327304 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3061eba-4b2b-40f2-bb0d-efaa71667945 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:26:51.333380 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb39e48-537d-470f-92db-151da6ede014 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:26:51.333976 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb39e48-537d-470f-92db-151da6ede014 service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:26:51.334252 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb39e48-537d-470f-92db-151da6ede014 service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:26:51.334520 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb39e48-537d-470f-92db-151da6ede014 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:26:51.336197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-fdb39e48-537d-470f-92db-151da6ede014 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:26:51.394876 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb39e48-537d-470f-92db-151da6ede014 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:26:51.396503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb39e48-537d-470f-92db-151da6ede014 service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:26:51.405612 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1211/2413] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:26:51 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:27:01.401221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=28 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:00.606617 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9971a4-08f7-439a-95aa-64445dca3850 service swift] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:28:00.619178 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9971a4-08f7-439a-95aa-64445dca3850 service swift] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:28:00.625402 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335ed90-a2d7-43dd-91af-70310bf6abec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:28:00.625998 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335ed90-a2d7-43dd-91af-70310bf6abec service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:00.626279 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335ed90-a2d7-43dd-91af-70310bf6abec service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:00.626552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335ed90-a2d7-43dd-91af-70310bf6abec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:00.628262 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-8335ed90-a2d7-43dd-91af-70310bf6abec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:00.687839 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8335ed90-a2d7-43dd-91af-70310bf6abec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:28:00.689429 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8335ed90-a2d7-43dd-91af-70310bf6abec service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:28:00.698986 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1203/2414] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:28:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:28:10.694787 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=28 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:28:14.355579 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7814d42-ed94-4366-b1ea-08ef5dd4f7fb None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:28:14.356105 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7814d42-ed94-4366-b1ea-08ef5dd4f7fb None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:28:14.356105 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7814d42-ed94-4366-b1ea-08ef5dd4f7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:28:14.376725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-e7814d42-ed94-4366-b1ea-08ef5dd4f7fb None None] MFA Rules not processed for user `3ff39cd0e766400f9d25e3ad465c4faf`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:28:14.377928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e7814d42-ed94-4366-b1ea-08ef5dd4f7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:28:14.384816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1212/2415] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:28:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:28:24.369421 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:29:41.115947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335ed90-a2d7-43dd-91af-70310bf6abec service nova] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:29:41.176890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335ed90-a2d7-43dd-91af-70310bf6abec service nova] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:29:41.181900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d273dd-a600-4bd4-a142-4118a22e9ac0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdacdcb2451742938164fdd44cdbaebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:29:41.182500 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d273dd-a600-4bd4-a142-4118a22e9ac0 service placement] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:41.182781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d273dd-a600-4bd4-a142-4118a22e9ac0 service placement] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:41.183085 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d273dd-a600-4bd4-a142-4118a22e9ac0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:41.184823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-19d273dd-a600-4bd4-a142-4118a22e9ac0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:41.217208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d273dd-a600-4bd4-a142-4118a22e9ac0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:29:41.218848 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d273dd-a600-4bd4-a142-4118a22e9ac0 service placement] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:29:41.229183 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1204/2416] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:29:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:29:51.225105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=36 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:29:57.482355 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7814d42-ed94-4366-b1ea-08ef5dd4f7fb None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:29:57.483763 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e7814d42-ed94-4366-b1ea-08ef5dd4f7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:57.574796 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7814d42-ed94-4366-b1ea-08ef5dd4f7fb None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:29:57.578463 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c8e69-c7ef-4203-96e0-6517c0e3bf68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:29:57.579091 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c8e69-c7ef-4203-96e0-6517c0e3bf68 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:29:57.579372 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c8e69-c7ef-4203-96e0-6517c0e3bf68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:29:57.579642 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c8e69-c7ef-4203-96e0-6517c0e3bf68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:29:57.581378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-244c8e69-c7ef-4203-96e0-6517c0e3bf68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:29:57.592974 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c8e69-c7ef-4203-96e0-6517c0e3bf68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:29:57.594395 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c8e69-c7ef-4203-96e0-6517c0e3bf68 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:29:57.603412 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1213/2417] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:29:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:30:03.019356 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d273dd-a600-4bd4-a142-4118a22e9ac0 service placement] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:30:03.050213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d273dd-a600-4bd4-a142-4118a22e9ac0 service placement] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:30:03.056761 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a22b8-6e9a-46f5-adf1-3bfda0ad206a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:30:03.057371 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a22b8-6e9a-46f5-adf1-3bfda0ad206a service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:30:03.057651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a22b8-6e9a-46f5-adf1-3bfda0ad206a service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:30:03.057922 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a22b8-6e9a-46f5-adf1-3bfda0ad206a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:30:03.059673 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c00a22b8-6e9a-46f5-adf1-3bfda0ad206a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:30:03.085021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a22b8-6e9a-46f5-adf1-3bfda0ad206a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:30:03.086794 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a22b8-6e9a-46f5-adf1-3bfda0ad206a service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:30:03.097407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1205/2418] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:30:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:30:07.599352 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=41 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:30:13.092924 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=18 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:30:28.767336 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c8e69-c7ef-4203-96e0-6517c0e3bf68 service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:30:28.787054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c8e69-c7ef-4203-96e0-6517c0e3bf68 service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:30:28.790842 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823bd7a-9b31-46f7-b8fd-489c6c914567 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:30:28.791462 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823bd7a-9b31-46f7-b8fd-489c6c914567 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:30:28.791739 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823bd7a-9b31-46f7-b8fd-489c6c914567 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:30:28.792007 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823bd7a-9b31-46f7-b8fd-489c6c914567 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:30:28.802091 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823bd7a-9b31-46f7-b8fd-489c6c914567 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:30:28.803548 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823bd7a-9b31-46f7-b8fd-489c6c914567 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:30:28.814093 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1214/2419] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:30:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:30:38.809352 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=10 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:30:57.899889 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a22b8-6e9a-46f5-adf1-3bfda0ad206a service nova] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:30:57.911765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a22b8-6e9a-46f5-adf1-3bfda0ad206a service nova] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:30:57.917550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364e0da-00ea-4a0d-97ef-19b1d41612df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:30:57.918141 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364e0da-00ea-4a0d-97ef-19b1d41612df service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:30:57.918419 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364e0da-00ea-4a0d-97ef-19b1d41612df service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:30:57.918690 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364e0da-00ea-4a0d-97ef-19b1d41612df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:30:57.926534 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c364e0da-00ea-4a0d-97ef-19b1d41612df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:30:57.928293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c364e0da-00ea-4a0d-97ef-19b1d41612df service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:30:57.959112 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1206/2420] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:30:57 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:31:07.954069 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=13 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:31:47.763412 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823bd7a-9b31-46f7-b8fd-489c6c914567 service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:31:47.772163 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823bd7a-9b31-46f7-b8fd-489c6c914567 service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:31:47.776066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d106a3-a642-4633-9f8a-53dbd04a4a9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:31:47.776654 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d106a3-a642-4633-9f8a-53dbd04a4a9f service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:31:47.776929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d106a3-a642-4633-9f8a-53dbd04a4a9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:31:47.777194 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d106a3-a642-4633-9f8a-53dbd04a4a9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:31:47.784265 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d106a3-a642-4633-9f8a-53dbd04a4a9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:31:47.785715 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d106a3-a642-4633-9f8a-53dbd04a4a9f service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:31:47.794176 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1215/2421] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:31:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:31:57.790712 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=6 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:31:57.901940 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364e0da-00ea-4a0d-97ef-19b1d41612df service nova] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:31:57.911270 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364e0da-00ea-4a0d-97ef-19b1d41612df service nova] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:31:57.915363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbedd4ac-c3e8-4066-9ecd-2d5885326374 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:31:57.915959 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbedd4ac-c3e8-4066-9ecd-2d5885326374 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:31:57.916236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbedd4ac-c3e8-4066-9ecd-2d5885326374 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:31:57.916507 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbedd4ac-c3e8-4066-9ecd-2d5885326374 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:31:57.924883 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbedd4ac-c3e8-4066-9ecd-2d5885326374 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:31:57.926310 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbedd4ac-c3e8-4066-9ecd-2d5885326374 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:31:57.953778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1207/2422] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:31:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:31:58.023674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d106a3-a642-4633-9f8a-53dbd04a4a9f service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:31:58.032993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d106a3-a642-4633-9f8a-53dbd04a4a9f service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:31:58.036721 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167baf2d-1844-4e1c-960f-f52c16ec41bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:31:58.037550 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167baf2d-1844-4e1c-960f-f52c16ec41bd service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:31:58.037879 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167baf2d-1844-4e1c-960f-f52c16ec41bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:31:58.038499 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167baf2d-1844-4e1c-960f-f52c16ec41bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:31:58.050480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167baf2d-1844-4e1c-960f-f52c16ec41bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:31:58.051906 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167baf2d-1844-4e1c-960f-f52c16ec41bd service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:31:58.064051 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1216/2423] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:31:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:31:58.237159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbedd4ac-c3e8-4066-9ecd-2d5885326374 service cinder] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:31:58.247481 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbedd4ac-c3e8-4066-9ecd-2d5885326374 service cinder] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:31:58.251192 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c518e-a1a9-4351-a6ba-f2cc8757eb3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:31:58.251798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c518e-a1a9-4351-a6ba-f2cc8757eb3f service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:31:58.252078 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c518e-a1a9-4351-a6ba-f2cc8757eb3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:31:58.252384 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c518e-a1a9-4351-a6ba-f2cc8757eb3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:31:58.262010 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c518e-a1a9-4351-a6ba-f2cc8757eb3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:31:58.263477 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c518e-a1a9-4351-a6ba-f2cc8757eb3f service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:31:58.291144 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1208/2424] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:31:58 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:32:08.059068 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=6 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:32:08.288540 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=26 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:33:58.502589 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167baf2d-1844-4e1c-960f-f52c16ec41bd service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:33:58.514337 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167baf2d-1844-4e1c-960f-f52c16ec41bd service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:33:58.520452 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe90c0d-4a07-4b08-90b0-e9315587a1d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:33:58.521063 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe90c0d-4a07-4b08-90b0-e9315587a1d4 service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:33:58.521343 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe90c0d-4a07-4b08-90b0-e9315587a1d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:33:58.521624 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe90c0d-4a07-4b08-90b0-e9315587a1d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:33:58.529828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe90c0d-4a07-4b08-90b0-e9315587a1d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:33:58.532416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe90c0d-4a07-4b08-90b0-e9315587a1d4 service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:33:58.561302 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1217/2425] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:33:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:34:08.558665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=13 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:34:46.034311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c518e-a1a9-4351-a6ba-f2cc8757eb3f service cinder] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:34:46.035740 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f73c518e-a1a9-4351-a6ba-f2cc8757eb3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:34:46.050631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c518e-a1a9-4351-a6ba-f2cc8757eb3f service cinder] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:34:46.055932 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e748d-3438-4d9b-adf3-8fadcc071864 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdacdcb2451742938164fdd44cdbaebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:34:46.056537 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e748d-3438-4d9b-adf3-8fadcc071864 service placement] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:34:46.056900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e748d-3438-4d9b-adf3-8fadcc071864 service placement] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:34:46.057178 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e748d-3438-4d9b-adf3-8fadcc071864 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:34:46.065797 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082e748d-3438-4d9b-adf3-8fadcc071864 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:34:46.067407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082e748d-3438-4d9b-adf3-8fadcc071864 service placement] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:34:46.077963 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1209/2426] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:34:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:34:56.073054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=6 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:35:58.644231 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe90c0d-4a07-4b08-90b0-e9315587a1d4 service nova] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:58.654002 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe90c0d-4a07-4b08-90b0-e9315587a1d4 service nova] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:58.657610 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b715679c-fc32-4cf6-ae1d-d902c130c20d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:58.658208 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715679c-fc32-4cf6-ae1d-d902c130c20d service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:58.658486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715679c-fc32-4cf6-ae1d-d902c130c20d service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:58.658756 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715679c-fc32-4cf6-ae1d-d902c130c20d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:58.660605 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b715679c-fc32-4cf6-ae1d-d902c130c20d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:58.729793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b715679c-fc32-4cf6-ae1d-d902c130c20d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:58.731292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b715679c-fc32-4cf6-ae1d-d902c130c20d service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:58.743417 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1218/2427] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:35:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:35:58.801880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e748d-3438-4d9b-adf3-8fadcc071864 service placement] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:35:58.811615 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e748d-3438-4d9b-adf3-8fadcc071864 service placement] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:35:58.815275 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c0354-4823-4a67-ad72-5eb2af724b51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:35:58.815869 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c0354-4823-4a67-ad72-5eb2af724b51 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:35:58.816149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c0354-4823-4a67-ad72-5eb2af724b51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:35:58.816423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c0354-4823-4a67-ad72-5eb2af724b51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:35:58.818416 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-1e7c0354-4823-4a67-ad72-5eb2af724b51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:35:58.893370 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c0354-4823-4a67-ad72-5eb2af724b51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:35:58.894705 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c0354-4823-4a67-ad72-5eb2af724b51 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:35:58.903949 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1210/2428] 10.222.0.67 () {60 vars in 1396 bytes} [Wed Aug 31 00:35:58 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3266 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:36:08.737865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=34 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:08.899816 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=33 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:36:52.711141 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b715679c-fc32-4cf6-ae1d-d902c130c20d service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:36:52.712764 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b715679c-fc32-4cf6-ae1d-d902c130c20d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:52.779437 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b715679c-fc32-4cf6-ae1d-d902c130c20d service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:36:52.783050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29db1f-b388-4ef7-b5e1-bed7c3c39f80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:36:52.783662 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29db1f-b388-4ef7-b5e1-bed7c3c39f80 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:36:52.783942 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29db1f-b388-4ef7-b5e1-bed7c3c39f80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:36:52.784214 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29db1f-b388-4ef7-b5e1-bed7c3c39f80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:36:52.786549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-8c29db1f-b388-4ef7-b5e1-bed7c3c39f80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:36:52.797839 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c29db1f-b388-4ef7-b5e1-bed7c3c39f80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:36:52.800267 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c29db1f-b388-4ef7-b5e1-bed7c3c39f80 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:36:52.811219 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1219/2429] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:36:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:37:02.807335 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=31 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:37:58.976998 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c0354-4823-4a67-ad72-5eb2af724b51 service cinder] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:58.985978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c0354-4823-4a67-ad72-5eb2af724b51 service cinder] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:58.989609 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b58506-cb21-4f28-a64d-3a0081354541 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:58.990208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b58506-cb21-4f28-a64d-3a0081354541 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:58.990489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b58506-cb21-4f28-a64d-3a0081354541 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:58.990762 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b58506-cb21-4f28-a64d-3a0081354541 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:58.992742 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-60b58506-cb21-4f28-a64d-3a0081354541 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:59.069032 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b58506-cb21-4f28-a64d-3a0081354541 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:59.070461 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b58506-cb21-4f28-a64d-3a0081354541 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:59.080401 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1211/2430] 10.222.0.67 () {60 vars in 1396 bytes} [Wed Aug 31 00:37:58 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3270 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:37:59.129207 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29db1f-b388-4ef7-b5e1-bed7c3c39f80 service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:59.139156 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29db1f-b388-4ef7-b5e1-bed7c3c39f80 service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:59.143045 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcf397-f073-495b-af35-8b732364c298 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:59.143657 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcf397-f073-495b-af35-8b732364c298 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:59.143953 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcf397-f073-495b-af35-8b732364c298 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:59.144217 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcf397-f073-495b-af35-8b732364c298 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:59.146091 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b1fcf397-f073-495b-af35-8b732364c298 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:59.160296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fcf397-f073-495b-af35-8b732364c298 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:59.161792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fcf397-f073-495b-af35-8b732364c298 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:59.173490 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1220/2431] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:37:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:37:59.223526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b58506-cb21-4f28-a64d-3a0081354541 service cinder] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:59.225096 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-60b58506-cb21-4f28-a64d-3a0081354541 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:59.247220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b58506-cb21-4f28-a64d-3a0081354541 service cinder] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:59.264170 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b7356-f414-4593-901c-b3e8e6951ea4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:59.264170 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b7356-f414-4593-901c-b3e8e6951ea4 service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:59.264170 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b7356-f414-4593-901c-b3e8e6951ea4 service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:59.264170 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b7356-f414-4593-901c-b3e8e6951ea4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:59.268685 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcf397-f073-495b-af35-8b732364c298 service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:37:59.268685 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b1fcf397-f073-495b-af35-8b732364c298 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:59.284083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcf397-f073-495b-af35-8b732364c298 service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:37:59.291207 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9e2d7-c0a9-474a-b6a2-581718a81eb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:37:59.291823 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9e2d7-c0a9-474a-b6a2-581718a81eb8 service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:37:59.292102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9e2d7-c0a9-474a-b6a2-581718a81eb8 service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:37:59.292373 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9e2d7-c0a9-474a-b6a2-581718a81eb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:37:59.294283 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-96a9e2d7-c0a9-474a-b6a2-581718a81eb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:37:59.342034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b7356-f414-4593-901c-b3e8e6951ea4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:59.343656 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b7356-f414-4593-901c-b3e8e6951ea4 service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:59.362588 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1212/2432] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:37:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:37:59.366900 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a9e2d7-c0a9-474a-b6a2-581718a81eb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:37:59.368767 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a9e2d7-c0a9-474a-b6a2-581718a81eb8 service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:37:59.378302 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1221/2433] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:37:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:38:00.885319 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e634d-c27f-4d0b-93bd-b588fe88df42 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:38:00.885848 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e634d-c27f-4d0b-93bd-b588fe88df42 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:38:00.886228 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e634d-c27f-4d0b-93bd-b588fe88df42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:38:00.945510 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-6a3e634d-c27f-4d0b-93bd-b588fe88df42 None None] MFA Rules not processed for user `f9add734db2a4adc9857040166c02675`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:38:00.947963 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-6a3e634d-c27f-4d0b-93bd-b588fe88df42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:38:01.009225 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:38:01.020115 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1213/2434] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:38:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:38:01.570491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9e2d7-c0a9-474a-b6a2-581718a81eb8 service nova] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:38:01.580419 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9e2d7-c0a9-474a-b6a2-581718a81eb8 service nova] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:38:01.584031 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52739bdd-0c9f-40b7-b24d-50016ed0f7c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:38:01.584631 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52739bdd-0c9f-40b7-b24d-50016ed0f7c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:38:01.584912 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52739bdd-0c9f-40b7-b24d-50016ed0f7c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:38:01.585182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52739bdd-0c9f-40b7-b24d-50016ed0f7c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:38:01.587845 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-52739bdd-0c9f-40b7-b24d-50016ed0f7c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:38:01.597634 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52739bdd-0c9f-40b7-b24d-50016ed0f7c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:38:01.599079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52739bdd-0c9f-40b7-b24d-50016ed0f7c6 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:38:01.608006 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1222/2435] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:38:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:38:11.013633 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:38:11.604242 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=45 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:38:57.614667 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e634d-c27f-4d0b-93bd-b588fe88df42 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:38:57.624975 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e634d-c27f-4d0b-93bd-b588fe88df42 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:38:57.628567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8cdcf-2c0b-4953-b0b1-3bfeb2912996 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:38:57.629166 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8cdcf-2c0b-4953-b0b1-3bfeb2912996 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:38:57.629441 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8cdcf-2c0b-4953-b0b1-3bfeb2912996 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:38:57.629714 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8cdcf-2c0b-4953-b0b1-3bfeb2912996 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:38:57.631660 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-0ef8cdcf-2c0b-4953-b0b1-3bfeb2912996 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:38:57.643829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8cdcf-2c0b-4953-b0b1-3bfeb2912996 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:38:57.645244 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8cdcf-2c0b-4953-b0b1-3bfeb2912996 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:38:57.654530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1214/2436] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:38:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:39:07.650339 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=6 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:39:52.025776 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52739bdd-0c9f-40b7-b24d-50016ed0f7c6 service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:39:52.027411 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-52739bdd-0c9f-40b7-b24d-50016ed0f7c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:39:52.155820 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52739bdd-0c9f-40b7-b24d-50016ed0f7c6 service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:39:52.161000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ce17d-93c8-4cdf-a345-414f97f44b75 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdacdcb2451742938164fdd44cdbaebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:39:52.161604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ce17d-93c8-4cdf-a345-414f97f44b75 service placement] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:39:52.161884 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ce17d-93c8-4cdf-a345-414f97f44b75 service placement] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:39:52.162156 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ce17d-93c8-4cdf-a345-414f97f44b75 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:39:52.164128 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-948ce17d-93c8-4cdf-a345-414f97f44b75 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:39:52.175679 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ce17d-93c8-4cdf-a345-414f97f44b75 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:39:52.177261 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ce17d-93c8-4cdf-a345-414f97f44b75 service placement] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:39:52.187751 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1223/2437] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:39:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:40:02.183523 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=57 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:40:59.427372 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8cdcf-2c0b-4953-b0b1-3bfeb2912996 service cinder] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:40:59.491507 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8cdcf-2c0b-4953-b0b1-3bfeb2912996 service cinder] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:40:59.497781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa97af-b1cd-4986-98db-6c7cf8d6b054 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:40:59.498402 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa97af-b1cd-4986-98db-6c7cf8d6b054 service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:40:59.498681 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa97af-b1cd-4986-98db-6c7cf8d6b054 service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:40:59.498950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa97af-b1cd-4986-98db-6c7cf8d6b054 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:40:59.509341 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa97af-b1cd-4986-98db-6c7cf8d6b054 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:40:59.510865 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa97af-b1cd-4986-98db-6c7cf8d6b054 service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:40:59.544996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1215/2438] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:40:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:41:09.540214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=31 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:41:57.685966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ce17d-93c8-4cdf-a345-414f97f44b75 service placement] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:41:57.708696 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ce17d-93c8-4cdf-a345-414f97f44b75 service placement] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:41:57.712707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62258f4c-f3f7-485c-b3cf-7a9635d762ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:41:57.713326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62258f4c-f3f7-485c-b3cf-7a9635d762ba service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:41:57.713608 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62258f4c-f3f7-485c-b3cf-7a9635d762ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:41:57.713882 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62258f4c-f3f7-485c-b3cf-7a9635d762ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:41:57.721996 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62258f4c-f3f7-485c-b3cf-7a9635d762ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:41:57.723477 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62258f4c-f3f7-485c-b3cf-7a9635d762ba service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:41:57.733021 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1224/2439] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:41:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:42:07.728906 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=12 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:42:57.674329 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa97af-b1cd-4986-98db-6c7cf8d6b054 service nova] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:57.685930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa97af-b1cd-4986-98db-6c7cf8d6b054 service nova] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:57.691398 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe488a-cb14-4510-9fca-7c83288a98b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:57.691988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe488a-cb14-4510-9fca-7c83288a98b6 service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:57.692265 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe488a-cb14-4510-9fca-7c83288a98b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:57.692535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe488a-cb14-4510-9fca-7c83288a98b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:57.700388 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe488a-cb14-4510-9fca-7c83288a98b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:57.701943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe488a-cb14-4510-9fca-7c83288a98b6 service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:57.730363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1216/2440] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:42:57 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:42:59.093255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62258f4c-f3f7-485c-b3cf-7a9635d762ba service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:59.094663 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-62258f4c-f3f7-485c-b3cf-7a9635d762ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:59.184578 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62258f4c-f3f7-485c-b3cf-7a9635d762ba service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:59.188402 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911d014-12b1-428c-ae33-f921c94c8925 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:59.189005 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911d014-12b1-428c-ae33-f921c94c8925 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:59.189287 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911d014-12b1-428c-ae33-f921c94c8925 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:59.189562 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911d014-12b1-428c-ae33-f921c94c8925 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:59.198856 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911d014-12b1-428c-ae33-f921c94c8925 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:59.200325 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911d014-12b1-428c-ae33-f921c94c8925 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:59.227837 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1225/2441] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:42:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:42:59.393194 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe488a-cb14-4510-9fca-7c83288a98b6 service nova] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:42:59.402587 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe488a-cb14-4510-9fca-7c83288a98b6 service nova] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:42:59.406233 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906055a-be41-45bf-80b1-26d2ae07076d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:42:59.406817 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906055a-be41-45bf-80b1-26d2ae07076d service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:42:59.407125 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906055a-be41-45bf-80b1-26d2ae07076d service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:42:59.407399 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906055a-be41-45bf-80b1-26d2ae07076d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:42:59.409173 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-4906055a-be41-45bf-80b1-26d2ae07076d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:42:59.433593 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4906055a-be41-45bf-80b1-26d2ae07076d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:42:59.435005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4906055a-be41-45bf-80b1-26d2ae07076d service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:42:59.443854 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1217/2442] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:42:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:43:09.224149 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=46 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:09.439802 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=24 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:43:58.594029 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911d014-12b1-428c-ae33-f921c94c8925 service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:43:58.602626 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911d014-12b1-428c-ae33-f921c94c8925 service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:43:58.607335 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affcdd91-6f85-464c-84f2-2e945f801b33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:43:58.607938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcdd91-6f85-464c-84f2-2e945f801b33 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:43:58.608218 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcdd91-6f85-464c-84f2-2e945f801b33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:43:58.608493 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcdd91-6f85-464c-84f2-2e945f801b33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:43:58.617295 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affcdd91-6f85-464c-84f2-2e945f801b33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:43:58.618723 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affcdd91-6f85-464c-84f2-2e945f801b33 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:43:58.646056 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1226/2443] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:43:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:44:02.166206 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906055a-be41-45bf-80b1-26d2ae07076d service cinder] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:02.175832 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906055a-be41-45bf-80b1-26d2ae07076d service cinder] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:02.179543 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675de1d-a00f-46a3-87c5-d492eb98bc5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:02.180151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675de1d-a00f-46a3-87c5-d492eb98bc5c service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:02.180433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675de1d-a00f-46a3-87c5-d492eb98bc5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:02.180704 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675de1d-a00f-46a3-87c5-d492eb98bc5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:02.192878 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675de1d-a00f-46a3-87c5-d492eb98bc5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:02.194363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675de1d-a00f-46a3-87c5-d492eb98bc5c service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:02.206728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1218/2444] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:44:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:44:03.279186 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affcdd91-6f85-464c-84f2-2e945f801b33 service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:03.280814 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-affcdd91-6f85-464c-84f2-2e945f801b33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:03.296285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affcdd91-6f85-464c-84f2-2e945f801b33 service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:03.300032 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ae139-8213-4c10-bb9f-19ee75014c43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:03.300690 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ae139-8213-4c10-bb9f-19ee75014c43 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:03.300976 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ae139-8213-4c10-bb9f-19ee75014c43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:03.301462 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ae139-8213-4c10-bb9f-19ee75014c43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:03.304112 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-479ae139-8213-4c10-bb9f-19ee75014c43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:03.392263 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ae139-8213-4c10-bb9f-19ee75014c43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:03.393712 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ae139-8213-4c10-bb9f-19ee75014c43 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:03.412798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1227/2445] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:44:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:44:04.671003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675de1d-a00f-46a3-87c5-d492eb98bc5c service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:04.679762 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675de1d-a00f-46a3-87c5-d492eb98bc5c service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:04.684254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7c1c4-7ec0-4266-b9ac-ac383f07f3e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:04.684849 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7c1c4-7ec0-4266-b9ac-ac383f07f3e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:04.685126 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7c1c4-7ec0-4266-b9ac-ac383f07f3e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:04.685394 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7c1c4-7ec0-4266-b9ac-ac383f07f3e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:04.695923 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce7c1c4-7ec0-4266-b9ac-ac383f07f3e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:04.697473 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce7c1c4-7ec0-4266-b9ac-ac383f07f3e7 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:04.708951 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1219/2446] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:44:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:44:06.663556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ae139-8213-4c10-bb9f-19ee75014c43 service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:06.681240 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ae139-8213-4c10-bb9f-19ee75014c43 service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:06.688871 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b64aa-8112-43e1-acd5-ae8d0a48aa42 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdacdcb2451742938164fdd44cdbaebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:06.689548 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b64aa-8112-43e1-acd5-ae8d0a48aa42 service placement] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:06.690080 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b64aa-8112-43e1-acd5-ae8d0a48aa42 service placement] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:06.690995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b64aa-8112-43e1-acd5-ae8d0a48aa42 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:06.694275 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-627b64aa-8112-43e1-acd5-ae8d0a48aa42 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:06.708155 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b64aa-8112-43e1-acd5-ae8d0a48aa42 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:06.710932 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b64aa-8112-43e1-acd5-ae8d0a48aa42 service placement] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:06.724111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1228/2447] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:44:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:44:10.217126 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7c1c4-7ec0-4266-b9ac-ac383f07f3e7 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:10.229594 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7c1c4-7ec0-4266-b9ac-ac383f07f3e7 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:10.235354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea6c93-b986-44c2-b3a8-e06c0e126d2a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdacdcb2451742938164fdd44cdbaebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:10.235971 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea6c93-b986-44c2-b3a8-e06c0e126d2a service placement] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:10.236276 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea6c93-b986-44c2-b3a8-e06c0e126d2a service placement] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:10.236578 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea6c93-b986-44c2-b3a8-e06c0e126d2a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:10.238415 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f2ea6c93-b986-44c2-b3a8-e06c0e126d2a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:10.251388 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea6c93-b986-44c2-b3a8-e06c0e126d2a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:10.253124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea6c93-b986-44c2-b3a8-e06c0e126d2a service placement] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:10.265377 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1220/2448] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:44:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:44:10.278887 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b64aa-8112-43e1-acd5-ae8d0a48aa42 service placement] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:10.281439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-627b64aa-8112-43e1-acd5-ae8d0a48aa42 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:10.296650 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b64aa-8112-43e1-acd5-ae8d0a48aa42 service placement] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:10.301565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48623a4b-5eca-43ad-b45f-4810fe3d03d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:10.302142 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48623a4b-5eca-43ad-b45f-4810fe3d03d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:10.302427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48623a4b-5eca-43ad-b45f-4810fe3d03d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:10.302806 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48623a4b-5eca-43ad-b45f-4810fe3d03d9 admin admin] PATH_INFO: `/v3/users/f9add734db2a4adc9857040166c02675` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:10.304010 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48623a4b-5eca-43ad-b45f-4810fe3d03d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9add734db2a4adc9857040166c02675)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:10.305978 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48623a4b-5eca-43ad-b45f-4810fe3d03d9 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:10.350984 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-48623a4b-5eca-43ad-b45f-4810fe3d03d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9add734db2a4adc9857040166c02675'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:10.358370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-48623a4b-5eca-43ad-b45f-4810fe3d03d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9add734db2a4adc9857040166c02675'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:10.365245 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-48623a4b-5eca-43ad-b45f-4810fe3d03d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9add734db2a4adc9857040166c02675'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:10.365909 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-48623a4b-5eca-43ad-b45f-4810fe3d03d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9add734db2a4adc9857040166c02675'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:10.366537 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-48623a4b-5eca-43ad-b45f-4810fe3d03d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9add734db2a4adc9857040166c02675'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:10.379056 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:44:10.387657 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1229/2449] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:44:10 2022] DELETE /identity/v3/users/f9add734db2a4adc9857040166c02675 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:44:10.560272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea6c93-b986-44c2-b3a8-e06c0e126d2a service placement] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:10.570771 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea6c93-b986-44c2-b3a8-e06c0e126d2a service placement] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:10.631062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcac44f-4e1f-4d1a-9c58-338e1c8aa26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:10.631718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcac44f-4e1f-4d1a-9c58-338e1c8aa26e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:10.631998 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcac44f-4e1f-4d1a-9c58-338e1c8aa26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:10.632266 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcac44f-4e1f-4d1a-9c58-338e1c8aa26e admin admin] PATH_INFO: `/v3/projects/d201c4183dde4ac693ac92c43c02df77` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:10.634210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcac44f-4e1f-4d1a-9c58-338e1c8aa26e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d201c4183dde4ac693ac92c43c02df77)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:10.635414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcac44f-4e1f-4d1a-9c58-338e1c8aa26e admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:10.679833 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-1fcac44f-4e1f-4d1a-9c58-338e1c8aa26e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd201c4183dde4ac693ac92c43c02df77', 'request_id': 'req-1fcac44f-4e1f-4d1a-9c58-338e1c8aa26e'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:10.687501 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-1fcac44f-4e1f-4d1a-9c58-338e1c8aa26e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd201c4183dde4ac693ac92c43c02df77', 'request_id': 'req-1fcac44f-4e1f-4d1a-9c58-338e1c8aa26e'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:10.693003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-1fcac44f-4e1f-4d1a-9c58-338e1c8aa26e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd201c4183dde4ac693ac92c43c02df77', 'request_id': 'req-1fcac44f-4e1f-4d1a-9c58-338e1c8aa26e'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:10.703734 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-1fcac44f-4e1f-4d1a-9c58-338e1c8aa26e admin admin] The token cache is being invalidate because project d201c4183dde4ac693ac92c43c02df77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:44:10.704017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-1fcac44f-4e1f-4d1a-9c58-338e1c8aa26e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:10.704649 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-1fcac44f-4e1f-4d1a-9c58-338e1c8aa26e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:10.706142 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1221/2450] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:44:10 2022] DELETE /identity/v3/projects/d201c4183dde4ac693ac92c43c02df77 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:44:10.710092 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48623a4b-5eca-43ad-b45f-4810fe3d03d9 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:10.711665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-48623a4b-5eca-43ad-b45f-4810fe3d03d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:10.725782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48623a4b-5eca-43ad-b45f-4810fe3d03d9 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:10.783255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d2423-b0f7-4717-baeb-f434d3a74707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:10.783878 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d2423-b0f7-4717-baeb-f434d3a74707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:10.784154 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d2423-b0f7-4717-baeb-f434d3a74707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:10.784424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d2423-b0f7-4717-baeb-f434d3a74707 admin admin] PATH_INFO: `/v3/users/61ba3008e5284162996252a821103d9a` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:10.797830 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d2423-b0f7-4717-baeb-f434d3a74707 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61ba3008e5284162996252a821103d9a)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:10.799786 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d2423-b0f7-4717-baeb-f434d3a74707 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:10.847520 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-b41d2423-b0f7-4717-baeb-f434d3a74707 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61ba3008e5284162996252a821103d9a'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:10.854934 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-b41d2423-b0f7-4717-baeb-f434d3a74707 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61ba3008e5284162996252a821103d9a'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:10.862427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-b41d2423-b0f7-4717-baeb-f434d3a74707 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61ba3008e5284162996252a821103d9a'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:10.863146 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-b41d2423-b0f7-4717-baeb-f434d3a74707 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61ba3008e5284162996252a821103d9a'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:10.863770 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-b41d2423-b0f7-4717-baeb-f434d3a74707 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61ba3008e5284162996252a821103d9a'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:10.881520 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1230/2451] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:44:10 2022] DELETE /identity/v3/users/61ba3008e5284162996252a821103d9a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:44:11.065546 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcac44f-4e1f-4d1a-9c58-338e1c8aa26e admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:11.066981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-1fcac44f-4e1f-4d1a-9c58-338e1c8aa26e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:11.080604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcac44f-4e1f-4d1a-9c58-338e1c8aa26e admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:11.129330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:44:11.141216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3001c-eec9-43ee-b770-1526c81eb851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:11.141852 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3001c-eec9-43ee-b770-1526c81eb851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:11.142140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3001c-eec9-43ee-b770-1526c81eb851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:11.142413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3001c-eec9-43ee-b770-1526c81eb851 admin admin] PATH_INFO: `/v3/projects/e99f9b4e4d974334b90fcb122bdc2342` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:11.153039 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3001c-eec9-43ee-b770-1526c81eb851 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e99f9b4e4d974334b90fcb122bdc2342)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:11.154956 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3001c-eec9-43ee-b770-1526c81eb851 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:11.201019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e7c3001c-eec9-43ee-b770-1526c81eb851 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e99f9b4e4d974334b90fcb122bdc2342', 'request_id': 'req-e7c3001c-eec9-43ee-b770-1526c81eb851'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:11.209028 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e7c3001c-eec9-43ee-b770-1526c81eb851 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e99f9b4e4d974334b90fcb122bdc2342', 'request_id': 'req-e7c3001c-eec9-43ee-b770-1526c81eb851'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:11.216441 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e7c3001c-eec9-43ee-b770-1526c81eb851 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e99f9b4e4d974334b90fcb122bdc2342', 'request_id': 'req-e7c3001c-eec9-43ee-b770-1526c81eb851'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:11.229782 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e7c3001c-eec9-43ee-b770-1526c81eb851 admin admin] The token cache is being invalidate because project e99f9b4e4d974334b90fcb122bdc2342 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:44:11.230393 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e7c3001c-eec9-43ee-b770-1526c81eb851 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:11.230720 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e7c3001c-eec9-43ee-b770-1526c81eb851 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:44:11.232520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1222/2452] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:44:11 2022] DELETE /identity/v3/projects/e99f9b4e4d974334b90fcb122bdc2342 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:44:20.877973 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=60,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:44:21.226332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=32,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:44:57.004526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d2423-b0f7-4717-baeb-f434d3a74707 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:44:57.011477 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b41d2423-b0f7-4717-baeb-f434d3a74707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:57.074065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d2423-b0f7-4717-baeb-f434d3a74707 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:44:57.135006 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4a417-dc8a-4653-9756-e3dcdf026f6d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdacdcb2451742938164fdd44cdbaebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:44:57.136259 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4a417-dc8a-4653-9756-e3dcdf026f6d service placement] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:44:57.136575 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4a417-dc8a-4653-9756-e3dcdf026f6d service placement] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:44:57.136852 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4a417-dc8a-4653-9756-e3dcdf026f6d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:44:57.138625 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-ccb4a417-dc8a-4653-9756-e3dcdf026f6d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:44:57.150300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4a417-dc8a-4653-9756-e3dcdf026f6d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:44:57.152589 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4a417-dc8a-4653-9756-e3dcdf026f6d service placement] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:44:57.163069 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1231/2453] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:44:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:45:07.158443 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=58 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:45:59.840453 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3001c-eec9-43ee-b770-1526c81eb851 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:59.841859 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-e7c3001c-eec9-43ee-b770-1526c81eb851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:59.900185 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3001c-eec9-43ee-b770-1526c81eb851 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:45:59.903845 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e1d36-4f17-4d13-9ba5-c17b83169cc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:45:59.904565 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e1d36-4f17-4d13-9ba5-c17b83169cc2 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:45:59.904849 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e1d36-4f17-4d13-9ba5-c17b83169cc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:45:59.905126 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e1d36-4f17-4d13-9ba5-c17b83169cc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:45:59.907727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-5d1e1d36-4f17-4d13-9ba5-c17b83169cc2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:59.981185 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e1d36-4f17-4d13-9ba5-c17b83169cc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:45:59.982736 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e1d36-4f17-4d13-9ba5-c17b83169cc2 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:45:59.983383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4a417-dc8a-4653-9756-e3dcdf026f6d service placement] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:45:59.985660 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-ccb4a417-dc8a-4653-9756-e3dcdf026f6d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:45:59.995941 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1223/2454] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:45:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:45:59.997174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4a417-dc8a-4653-9756-e3dcdf026f6d service placement] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:46:00.001131 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e171713-f24c-4cef-9639-f31541968731 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:46:00.001804 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e171713-f24c-4cef-9639-f31541968731 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:46:00.002156 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e171713-f24c-4cef-9639-f31541968731 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:46:00.002516 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e171713-f24c-4cef-9639-f31541968731 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:46:00.011365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e171713-f24c-4cef-9639-f31541968731 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:46:00.012793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e171713-f24c-4cef-9639-f31541968731 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:46:00.026307 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1232/2455] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:45:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:46:02.560696 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e1d36-4f17-4d13-9ba5-c17b83169cc2 service cinder] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:46:02.562947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-5d1e1d36-4f17-4d13-9ba5-c17b83169cc2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:46:02.690792 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e1d36-4f17-4d13-9ba5-c17b83169cc2 service cinder] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:46:02.697813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2268a8f-ab60-4895-8421-cbfab23d51cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:46:02.698791 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2268a8f-ab60-4895-8421-cbfab23d51cc service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:46:02.699220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2268a8f-ab60-4895-8421-cbfab23d51cc service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:46:02.699568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2268a8f-ab60-4895-8421-cbfab23d51cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:46:02.704495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c2268a8f-ab60-4895-8421-cbfab23d51cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:46:02.722566 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2268a8f-ab60-4895-8421-cbfab23d51cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:46:02.725266 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2268a8f-ab60-4895-8421-cbfab23d51cc service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:46:02.750221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1224/2456] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:46:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:46:10.019844 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=6 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:46:12.732639 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=92 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:47:05.709799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e171713-f24c-4cef-9639-f31541968731 service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:47:05.711248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4e171713-f24c-4cef-9639-f31541968731 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:47:05.780492 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e171713-f24c-4cef-9639-f31541968731 service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:47:05.784242 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc9820-c403-4256-85f6-4fa314c4d79a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:47:05.785398 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc9820-c403-4256-85f6-4fa314c4d79a service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:47:05.785854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc9820-c403-4256-85f6-4fa314c4d79a service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:47:05.785991 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc9820-c403-4256-85f6-4fa314c4d79a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:47:05.787706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-3ebc9820-c403-4256-85f6-4fa314c4d79a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:47:05.797155 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc9820-c403-4256-85f6-4fa314c4d79a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:47:05.798572 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc9820-c403-4256-85f6-4fa314c4d79a service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:47:05.806466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1233/2457] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:47:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:47:15.802950 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=31 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:47:59.734667 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2268a8f-ab60-4895-8421-cbfab23d51cc service nova] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:47:59.736306 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c2268a8f-ab60-4895-8421-cbfab23d51cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:47:59.750531 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2268a8f-ab60-4895-8421-cbfab23d51cc service nova] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:47:59.756052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f27ebb-f6c7-467e-987a-21b7e23b0291 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:47:59.756639 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f27ebb-f6c7-467e-987a-21b7e23b0291 service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:47:59.757590 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f27ebb-f6c7-467e-987a-21b7e23b0291 service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:47:59.757850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f27ebb-f6c7-467e-987a-21b7e23b0291 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:47:59.759555 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-26f27ebb-f6c7-467e-987a-21b7e23b0291 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:47:59.830294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f27ebb-f6c7-467e-987a-21b7e23b0291 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:47:59.832868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f27ebb-f6c7-467e-987a-21b7e23b0291 service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:47:59.852154 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1225/2458] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:47:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:48:00.062911 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc9820-c403-4256-85f6-4fa314c4d79a service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:48:00.072864 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc9820-c403-4256-85f6-4fa314c4d79a service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:48:00.076582 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f330a-a0de-4629-8e7e-80cb92166756 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:48:00.077175 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f330a-a0de-4629-8e7e-80cb92166756 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:48:00.077450 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f330a-a0de-4629-8e7e-80cb92166756 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:48:00.077716 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f330a-a0de-4629-8e7e-80cb92166756 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:48:00.080326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-0a5f330a-a0de-4629-8e7e-80cb92166756 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:48:00.109067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f27ebb-f6c7-467e-987a-21b7e23b0291 service nova] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:48:00.125860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f27ebb-f6c7-467e-987a-21b7e23b0291 service nova] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:48:00.131687 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc1022-ff59-416d-bfeb-985bbe06889e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:48:00.132334 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc1022-ff59-416d-bfeb-985bbe06889e service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:48:00.132815 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc1022-ff59-416d-bfeb-985bbe06889e service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:48:00.134192 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc1022-ff59-416d-bfeb-985bbe06889e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:48:00.135057 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c6dc1022-ff59-416d-bfeb-985bbe06889e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:48:00.164606 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f330a-a0de-4629-8e7e-80cb92166756 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:48:00.166042 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f330a-a0de-4629-8e7e-80cb92166756 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:48:00.175432 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1234/2459] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:48:00 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:48:00.211768 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc1022-ff59-416d-bfeb-985bbe06889e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:48:00.213678 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc1022-ff59-416d-bfeb-985bbe06889e service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:48:00.222429 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1226/2460] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:48:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:48:10.171015 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=33 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:48:10.218166 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=69 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:49:02.235157 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f330a-a0de-4629-8e7e-80cb92166756 service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:49:02.245205 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f330a-a0de-4629-8e7e-80cb92166756 service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:49:02.248825 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761187c-2135-443b-868d-f9c150460a95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:49:02.249427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761187c-2135-443b-868d-f9c150460a95 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:49:02.249736 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761187c-2135-443b-868d-f9c150460a95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:49:02.250012 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761187c-2135-443b-868d-f9c150460a95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:49:02.251763 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-8761187c-2135-443b-868d-f9c150460a95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:49:02.265830 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761187c-2135-443b-868d-f9c150460a95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:49:02.267295 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761187c-2135-443b-868d-f9c150460a95 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:49:02.278826 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1235/2461] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:49:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:49:03.554895 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc1022-ff59-416d-bfeb-985bbe06889e service nova] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:49:03.564522 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc1022-ff59-416d-bfeb-985bbe06889e service nova] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:49:03.568173 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe64487-3f2e-4de7-8aab-e7b0b1254ffc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:49:03.568773 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe64487-3f2e-4de7-8aab-e7b0b1254ffc service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:49:03.569053 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe64487-3f2e-4de7-8aab-e7b0b1254ffc service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:49:03.569326 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe64487-3f2e-4de7-8aab-e7b0b1254ffc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:49:03.571046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-dfe64487-3f2e-4de7-8aab-e7b0b1254ffc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:49:03.585251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe64487-3f2e-4de7-8aab-e7b0b1254ffc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:49:03.586671 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe64487-3f2e-4de7-8aab-e7b0b1254ffc service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:49:03.598586 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1227/2462] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:49:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:49:12.274075 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=6 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:49:13.592718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=6 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:50:00.965107 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761187c-2135-443b-868d-f9c150460a95 service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:50:00.966511 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-8761187c-2135-443b-868d-f9c150460a95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:50:01.023335 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761187c-2135-443b-868d-f9c150460a95 service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:50:01.028552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75806d15-9eeb-4ede-b81c-39103c52efc5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdacdcb2451742938164fdd44cdbaebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:50:01.029150 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75806d15-9eeb-4ede-b81c-39103c52efc5 service placement] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:50:01.029432 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75806d15-9eeb-4ede-b81c-39103c52efc5 service placement] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:50:01.029705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75806d15-9eeb-4ede-b81c-39103c52efc5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:50:01.038462 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75806d15-9eeb-4ede-b81c-39103c52efc5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:50:01.040074 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75806d15-9eeb-4ede-b81c-39103c52efc5 service placement] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:50:01.050172 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1236/2463] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:50:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:50:11.045924 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=22 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:50:59.638543 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe64487-3f2e-4de7-8aab-e7b0b1254ffc service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:50:59.701958 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe64487-3f2e-4de7-8aab-e7b0b1254ffc service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:50:59.709317 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be302c-0e94-460f-bfb4-c6bd52557f6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:50:59.709474 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be302c-0e94-460f-bfb4-c6bd52557f6a service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:50:59.710442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be302c-0e94-460f-bfb4-c6bd52557f6a service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:50:59.710775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be302c-0e94-460f-bfb4-c6bd52557f6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:50:59.720101 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be302c-0e94-460f-bfb4-c6bd52557f6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:50:59.721699 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be302c-0e94-460f-bfb4-c6bd52557f6a service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:50:59.749454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1228/2464] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:50:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:51:09.746826 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=35 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:52:03.099782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64dd0a-2daa-43d1-a959-1b84b7683a90 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:52:03.100890 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64dd0a-2daa-43d1-a959-1b84b7683a90 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:52:03.100890 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64dd0a-2daa-43d1-a959-1b84b7683a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:52:03.129922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-ac64dd0a-2daa-43d1-a959-1b84b7683a90 None None] MFA Rules not processed for user `6d1a15d59d5d476ca6e0ab5176d310cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:52:03.131919 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-ac64dd0a-2daa-43d1-a959-1b84b7683a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:52:03.143400 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1237/2465] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:52:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:52:03.452004 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be302c-0e94-460f-bfb4-c6bd52557f6a service nova] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:52:03.474884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be302c-0e94-460f-bfb4-c6bd52557f6a service nova] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:52:03.479264 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966caac1-3487-4027-b7f6-af295f0a2ba2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:52:03.479862 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966caac1-3487-4027-b7f6-af295f0a2ba2 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:52:03.480138 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966caac1-3487-4027-b7f6-af295f0a2ba2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:52:03.480407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966caac1-3487-4027-b7f6-af295f0a2ba2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:52:03.482117 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-966caac1-3487-4027-b7f6-af295f0a2ba2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:52:03.492658 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966caac1-3487-4027-b7f6-af295f0a2ba2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:52:03.494062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966caac1-3487-4027-b7f6-af295f0a2ba2 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:52:03.502031 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1229/2466] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:52:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:52:04.882559 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64dd0a-2daa-43d1-a959-1b84b7683a90 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:52:04.894663 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64dd0a-2daa-43d1-a959-1b84b7683a90 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:52:04.900628 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf995f07-a803-4823-a8d8-e2fce5fcfc6d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdacdcb2451742938164fdd44cdbaebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:52:04.901241 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf995f07-a803-4823-a8d8-e2fce5fcfc6d service placement] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:52:04.901521 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf995f07-a803-4823-a8d8-e2fce5fcfc6d service placement] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:52:04.901793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf995f07-a803-4823-a8d8-e2fce5fcfc6d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:52:04.903831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-cf995f07-a803-4823-a8d8-e2fce5fcfc6d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:52:04.915664 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf995f07-a803-4823-a8d8-e2fce5fcfc6d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:52:04.917252 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf995f07-a803-4823-a8d8-e2fce5fcfc6d service placement] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:52:04.933058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1238/2467] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:52:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:52:10.300473 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966caac1-3487-4027-b7f6-af295f0a2ba2 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:52:10.310908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966caac1-3487-4027-b7f6-af295f0a2ba2 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:52:10.314617 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce3979-5354-4d06-9803-185177db7837 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:52:10.315252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce3979-5354-4d06-9803-185177db7837 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:52:10.315774 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce3979-5354-4d06-9803-185177db7837 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:52:10.316354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce3979-5354-4d06-9803-185177db7837 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:52:10.325195 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce3979-5354-4d06-9803-185177db7837 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:52:10.326823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce3979-5354-4d06-9803-185177db7837 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:52:10.335565 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1230/2468] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:52:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:52:11.141216 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf995f07-a803-4823-a8d8-e2fce5fcfc6d service placement] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:52:11.156629 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf995f07-a803-4823-a8d8-e2fce5fcfc6d service placement] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:52:11.163733 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02700275-298a-4b09-b261-51067d4cfd95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:52:11.164685 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02700275-298a-4b09-b261-51067d4cfd95 service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:52:11.164966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02700275-298a-4b09-b261-51067d4cfd95 service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:52:11.165408 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02700275-298a-4b09-b261-51067d4cfd95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:52:11.175671 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02700275-298a-4b09-b261-51067d4cfd95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:52:11.177331 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02700275-298a-4b09-b261-51067d4cfd95 service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:52:11.188988 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1239/2469] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:52:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:52:18.490355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce3979-5354-4d06-9803-185177db7837 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:52:18.504797 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce3979-5354-4d06-9803-185177db7837 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:52:18.511563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ca838-7534-4817-b25b-a8383a4982ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:52:18.512241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ca838-7534-4817-b25b-a8383a4982ad service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:52:18.512593 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ca838-7534-4817-b25b-a8383a4982ad service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:52:18.512924 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ca838-7534-4817-b25b-a8383a4982ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:52:18.514725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c60ca838-7534-4817-b25b-a8383a4982ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:52:18.528296 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ca838-7534-4817-b25b-a8383a4982ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:52:18.529879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ca838-7534-4817-b25b-a8383a4982ad service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:52:18.541549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1231/2470] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:52:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:52:21.184243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=20 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:52:22.881752 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02700275-298a-4b09-b261-51067d4cfd95 service nova] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:52:22.896099 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02700275-298a-4b09-b261-51067d4cfd95 service nova] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:52:22.905082 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16634aab-179b-4a7b-8270-08bff9ea3199 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:52:22.905082 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16634aab-179b-4a7b-8270-08bff9ea3199 service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:52:22.905082 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16634aab-179b-4a7b-8270-08bff9ea3199 service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:52:22.905082 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16634aab-179b-4a7b-8270-08bff9ea3199 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:52:22.905498 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-16634aab-179b-4a7b-8270-08bff9ea3199 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:52:22.919167 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16634aab-179b-4a7b-8270-08bff9ea3199 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:52:22.920773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16634aab-179b-4a7b-8270-08bff9ea3199 service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:52:22.932399 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1240/2471] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:52:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:52:28.537653 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=24 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:52:32.927078 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=6 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:52:58.481013 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ca838-7534-4817-b25b-a8383a4982ad service nova] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:52:58.490472 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ca838-7534-4817-b25b-a8383a4982ad service nova] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:52:58.494089 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857bb479-152d-45ee-9fc6-ffae2afc3746 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:52:58.494683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857bb479-152d-45ee-9fc6-ffae2afc3746 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:52:58.494969 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857bb479-152d-45ee-9fc6-ffae2afc3746 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:52:58.495286 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857bb479-152d-45ee-9fc6-ffae2afc3746 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:52:58.504147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857bb479-152d-45ee-9fc6-ffae2afc3746 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:52:58.505571 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857bb479-152d-45ee-9fc6-ffae2afc3746 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:52:58.534086 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1232/2472] 10.222.0.67 () {60 vars in 1396 bytes} [Wed Aug 31 00:52:58 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3270 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:53:08.530917 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=13 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:53:37.782193 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16634aab-179b-4a7b-8270-08bff9ea3199 service nova] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:53:37.795110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16634aab-179b-4a7b-8270-08bff9ea3199 service nova] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:53:37.800659 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c4fdd-30a1-46be-8f6d-89a4fe31edca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:53:37.801260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c4fdd-30a1-46be-8f6d-89a4fe31edca service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:53:37.801542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c4fdd-30a1-46be-8f6d-89a4fe31edca service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:53:37.801815 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c4fdd-30a1-46be-8f6d-89a4fe31edca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:53:37.810567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c4fdd-30a1-46be-8f6d-89a4fe31edca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:53:37.812262 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c4fdd-30a1-46be-8f6d-89a4fe31edca service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:53:37.839044 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1241/2473] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:53:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:53:47.836218 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=13 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:54:02.108618 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857bb479-152d-45ee-9fc6-ffae2afc3746 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:54:02.117456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857bb479-152d-45ee-9fc6-ffae2afc3746 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:54:02.121333 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67883aa-26f6-437a-9b8e-6a984f56cdde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:54:02.121937 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67883aa-26f6-437a-9b8e-6a984f56cdde service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:54:02.122220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67883aa-26f6-437a-9b8e-6a984f56cdde service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:54:02.122493 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67883aa-26f6-437a-9b8e-6a984f56cdde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:54:02.132732 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67883aa-26f6-437a-9b8e-6a984f56cdde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:54:02.134160 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67883aa-26f6-437a-9b8e-6a984f56cdde service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:54:02.145648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1233/2474] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:54:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:54:03.065787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c4fdd-30a1-46be-8f6d-89a4fe31edca service nova] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:54:03.086089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c4fdd-30a1-46be-8f6d-89a4fe31edca service nova] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:54:03.089813 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578a002-8b5f-456f-b3bd-c2466742ce0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:54:03.090409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578a002-8b5f-456f-b3bd-c2466742ce0d service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:54:03.090759 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578a002-8b5f-456f-b3bd-c2466742ce0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:54:03.091030 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578a002-8b5f-456f-b3bd-c2466742ce0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:54:03.099684 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1578a002-8b5f-456f-b3bd-c2466742ce0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:54:03.101104 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1578a002-8b5f-456f-b3bd-c2466742ce0d service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:54:03.110036 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1242/2475] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:54:03 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:54:04.110014 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67883aa-26f6-437a-9b8e-6a984f56cdde service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:54:04.118605 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67883aa-26f6-437a-9b8e-6a984f56cdde service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:54:04.123405 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232b52c-0035-4749-b43e-282e0b562818 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:54:04.124000 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232b52c-0035-4749-b43e-282e0b562818 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:54:04.124283 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232b52c-0035-4749-b43e-282e0b562818 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:54:04.124560 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232b52c-0035-4749-b43e-282e0b562818 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:54:04.134849 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232b52c-0035-4749-b43e-282e0b562818 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:54:04.136709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232b52c-0035-4749-b43e-282e0b562818 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:54:04.147806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1234/2476] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:54:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:54:13.106026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=10 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:54:14.142821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=12 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:55:04.980130 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578a002-8b5f-456f-b3bd-c2466742ce0d service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:55:05.054052 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578a002-8b5f-456f-b3bd-c2466742ce0d service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:55:05.059761 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a4b42-e19f-4ce3-8ec4-401981f7f164 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdacdcb2451742938164fdd44cdbaebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:55:05.060365 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a4b42-e19f-4ce3-8ec4-401981f7f164 service placement] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:55:05.060642 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a4b42-e19f-4ce3-8ec4-401981f7f164 service placement] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:55:05.060907 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a4b42-e19f-4ce3-8ec4-401981f7f164 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:55:05.062725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-911a4b42-e19f-4ce3-8ec4-401981f7f164 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:55:05.083366 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a4b42-e19f-4ce3-8ec4-401981f7f164 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:55:05.084926 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a4b42-e19f-4ce3-8ec4-401981f7f164 service placement] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:55:05.094607 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1243/2477] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:55:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:55:15.090555 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=36 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:55:59.819643 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232b52c-0035-4749-b43e-282e0b562818 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:55:59.831406 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232b52c-0035-4749-b43e-282e0b562818 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:55:59.836887 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be1c35-d06c-407a-bd06-2af63e4c24b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:55:59.837538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be1c35-d06c-407a-bd06-2af63e4c24b5 service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:55:59.837807 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be1c35-d06c-407a-bd06-2af63e4c24b5 service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:55:59.838077 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be1c35-d06c-407a-bd06-2af63e4c24b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:55:59.847465 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75be1c35-d06c-407a-bd06-2af63e4c24b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:55:59.849026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75be1c35-d06c-407a-bd06-2af63e4c24b5 service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:55:59.857827 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1235/2478] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:55:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:56:03.244098 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9d800-4449-4823-a3ac-40f6ae36bb3f None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:56:03.244634 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9d800-4449-4823-a3ac-40f6ae36bb3f None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:56:03.244634 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9d800-4449-4823-a3ac-40f6ae36bb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:56:03.286057 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-c0a9d800-4449-4823-a3ac-40f6ae36bb3f None None] MFA Rules not processed for user `7d7582f9cdd646f39404f7adc99ad9ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:56:03.287292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-c0a9d800-4449-4823-a3ac-40f6ae36bb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:56:03.344333 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1244/2479] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:56:03 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:56:09.853750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=6 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:56:13.337206 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=33 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:57:14.682420 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be1c35-d06c-407a-bd06-2af63e4c24b5 service nova] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:57:14.684048 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-75be1c35-d06c-407a-bd06-2af63e4c24b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:57:14.752399 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be1c35-d06c-407a-bd06-2af63e4c24b5 service nova] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:57:14.756994 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4bea7-56fb-422b-9b80-b517442cc2ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:57:14.757593 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4bea7-56fb-422b-9b80-b517442cc2ff service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:57:14.757875 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4bea7-56fb-422b-9b80-b517442cc2ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:57:14.758148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4bea7-56fb-422b-9b80-b517442cc2ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:57:14.759981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-26f4bea7-56fb-422b-9b80-b517442cc2ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:57:14.769565 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4bea7-56fb-422b-9b80-b517442cc2ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:57:14.771109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4bea7-56fb-422b-9b80-b517442cc2ff service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:57:14.779179 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1236/2480] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:57:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:57:24.775657 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=31 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:57:59.382363 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9d800-4449-4823-a3ac-40f6ae36bb3f None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:57:59.384278 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-c0a9d800-4449-4823-a3ac-40f6ae36bb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:57:59.463262 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9d800-4449-4823-a3ac-40f6ae36bb3f None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:57:59.468771 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd5e08-b963-47e5-a6a6-5d6001b2b316 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:57:59.469662 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd5e08-b963-47e5-a6a6-5d6001b2b316 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:57:59.469950 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd5e08-b963-47e5-a6a6-5d6001b2b316 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:57:59.470222 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd5e08-b963-47e5-a6a6-5d6001b2b316 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:57:59.481832 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd5e08-b963-47e5-a6a6-5d6001b2b316 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:57:59.483325 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd5e08-b963-47e5-a6a6-5d6001b2b316 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:57:59.493576 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1245/2481] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:57:59 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:58:03.857954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4bea7-56fb-422b-9b80-b517442cc2ff service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:03.860207 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-26f4bea7-56fb-422b-9b80-b517442cc2ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:04.010830 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4bea7-56fb-422b-9b80-b517442cc2ff service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:04.018313 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852ff01-982e-4fc5-b60b-7e40282e19ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:04.020198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852ff01-982e-4fc5-b60b-7e40282e19ab service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:04.020490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852ff01-982e-4fc5-b60b-7e40282e19ab service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:04.020776 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852ff01-982e-4fc5-b60b-7e40282e19ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:04.028142 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-4852ff01-982e-4fc5-b60b-7e40282e19ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:04.044622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4852ff01-982e-4fc5-b60b-7e40282e19ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:04.046293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4852ff01-982e-4fc5-b60b-7e40282e19ab service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:04.083699 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1237/2482] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:58:07.412341 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd5e08-b963-47e5-a6a6-5d6001b2b316 service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:07.414957 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-60bd5e08-b963-47e5-a6a6-5d6001b2b316 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:07.439330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd5e08-b963-47e5-a6a6-5d6001b2b316 service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:07.447217 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472116b-0c69-46b7-88d7-6d8a719764f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:07.447890 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472116b-0c69-46b7-88d7-6d8a719764f6 service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:07.448177 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472116b-0c69-46b7-88d7-6d8a719764f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:07.448700 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472116b-0c69-46b7-88d7-6d8a719764f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:07.461028 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472116b-0c69-46b7-88d7-6d8a719764f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:07.463006 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472116b-0c69-46b7-88d7-6d8a719764f6 service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:07.478699 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1246/2483] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:58:12.612888 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852ff01-982e-4fc5-b60b-7e40282e19ab service nova] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:12.614292 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-4852ff01-982e-4fc5-b60b-7e40282e19ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:12.626897 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852ff01-982e-4fc5-b60b-7e40282e19ab service nova] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:12.630420 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92efeceb-60d3-4f42-919e-17981d2ac4f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:12.631017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92efeceb-60d3-4f42-919e-17981d2ac4f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:12.631330 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92efeceb-60d3-4f42-919e-17981d2ac4f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:12.631606 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92efeceb-60d3-4f42-919e-17981d2ac4f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:12.633664 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-92efeceb-60d3-4f42-919e-17981d2ac4f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:12.718024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92efeceb-60d3-4f42-919e-17981d2ac4f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:12.719737 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92efeceb-60d3-4f42-919e-17981d2ac4f0 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:12.739086 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1238/2484] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:58:14.386208 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472116b-0c69-46b7-88d7-6d8a719764f6 service nova] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:14.396412 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472116b-0c69-46b7-88d7-6d8a719764f6 service nova] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:14.401557 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e367e80-51c4-45cb-a3f7-121917ce6fc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:14.402440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e367e80-51c4-45cb-a3f7-121917ce6fc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:14.402725 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e367e80-51c4-45cb-a3f7-121917ce6fc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:14.402999 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e367e80-51c4-45cb-a3f7-121917ce6fc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:14.405110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-8e367e80-51c4-45cb-a3f7-121917ce6fc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:14.418598 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e367e80-51c4-45cb-a3f7-121917ce6fc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:14.420256 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e367e80-51c4-45cb-a3f7-121917ce6fc5 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:14.433008 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1247/2485] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:58:15.028375 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92efeceb-60d3-4f42-919e-17981d2ac4f0 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:15.043630 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92efeceb-60d3-4f42-919e-17981d2ac4f0 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:15.055491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd90135-71d2-4093-860b-eb3423104860 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdacdcb2451742938164fdd44cdbaebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:15.056582 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd90135-71d2-4093-860b-eb3423104860 service placement] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:15.056863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd90135-71d2-4093-860b-eb3423104860 service placement] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:15.057293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd90135-71d2-4093-860b-eb3423104860 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:15.059373 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-4fd90135-71d2-4093-860b-eb3423104860 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:15.074781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd90135-71d2-4093-860b-eb3423104860 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:15.077186 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd90135-71d2-4093-860b-eb3423104860 service placement] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:15.114438 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e367e80-51c4-45cb-a3f7-121917ce6fc5 service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:15.116171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-8e367e80-51c4-45cb-a3f7-121917ce6fc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:15.118835 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1239/2486] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:58:15.156724 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e367e80-51c4-45cb-a3f7-121917ce6fc5 service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:15.161572 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757b70a-f102-41d5-afff-68b618f7c434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:15.162633 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757b70a-f102-41d5-afff-68b618f7c434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:15.164024 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757b70a-f102-41d5-afff-68b618f7c434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:15.164605 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757b70a-f102-41d5-afff-68b618f7c434 admin admin] PATH_INFO: `/v3/users/b03932102b7541f89a60a9c535a42e76` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:15.184191 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757b70a-f102-41d5-afff-68b618f7c434 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b03932102b7541f89a60a9c535a42e76)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:15.190865 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757b70a-f102-41d5-afff-68b618f7c434 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:15.249720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5757b70a-f102-41d5-afff-68b618f7c434 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b03932102b7541f89a60a9c535a42e76'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:15.259438 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5757b70a-f102-41d5-afff-68b618f7c434 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b03932102b7541f89a60a9c535a42e76'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:15.267178 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5757b70a-f102-41d5-afff-68b618f7c434 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b03932102b7541f89a60a9c535a42e76'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:15.267853 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5757b70a-f102-41d5-afff-68b618f7c434 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b03932102b7541f89a60a9c535a42e76'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:15.268456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-5757b70a-f102-41d5-afff-68b618f7c434 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b03932102b7541f89a60a9c535a42e76'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:15.287753 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1248/2487] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:58:15 2022] DELETE /identity/v3/users/b03932102b7541f89a60a9c535a42e76 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:58:15.472639 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd90135-71d2-4093-860b-eb3423104860 service placement] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:15.487664 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd90135-71d2-4093-860b-eb3423104860 service placement] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:15.527150 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:15.603303 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8dc0b4-13df-498a-b64e-efff0b98da17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:15.603960 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8dc0b4-13df-498a-b64e-efff0b98da17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:15.604450 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8dc0b4-13df-498a-b64e-efff0b98da17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:15.604563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8dc0b4-13df-498a-b64e-efff0b98da17 admin admin] PATH_INFO: `/v3/projects/e32cbe1a05ce414fb55f1de8bf71581c` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:15.622390 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8dc0b4-13df-498a-b64e-efff0b98da17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e32cbe1a05ce414fb55f1de8bf71581c)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:15.625374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8dc0b4-13df-498a-b64e-efff0b98da17 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:15.704360 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-7a8dc0b4-13df-498a-b64e-efff0b98da17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e32cbe1a05ce414fb55f1de8bf71581c', 'request_id': 'req-7a8dc0b4-13df-498a-b64e-efff0b98da17'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:15.715111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-7a8dc0b4-13df-498a-b64e-efff0b98da17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e32cbe1a05ce414fb55f1de8bf71581c', 'request_id': 'req-7a8dc0b4-13df-498a-b64e-efff0b98da17'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:15.724784 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-7a8dc0b4-13df-498a-b64e-efff0b98da17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e32cbe1a05ce414fb55f1de8bf71581c', 'request_id': 'req-7a8dc0b4-13df-498a-b64e-efff0b98da17'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:15.745128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-7a8dc0b4-13df-498a-b64e-efff0b98da17 admin admin] The token cache is being invalidate because project e32cbe1a05ce414fb55f1de8bf71581c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:58:15.745760 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-7a8dc0b4-13df-498a-b64e-efff0b98da17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:15.746605 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-7a8dc0b4-13df-498a-b64e-efff0b98da17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:15.750181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1240/2488] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:58:15 2022] DELETE /identity/v3/projects/e32cbe1a05ce414fb55f1de8bf71581c => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:15.755270 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600b83e-1618-420a-8efa-76493b5f7e5d None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:15.755736 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600b83e-1618-420a-8efa-76493b5f7e5d None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:15.756277 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600b83e-1618-420a-8efa-76493b5f7e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:15.826579 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:15.870157 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-a600b83e-1618-420a-8efa-76493b5f7e5d None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:15.871633 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a600b83e-1618-420a-8efa-76493b5f7e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:15.967801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1249/2489] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:58:15.975634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398271a1-47ae-439c-aa53-cf787b767e08 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:15.976035 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398271a1-47ae-439c-aa53-cf787b767e08 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:15.977119 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398271a1-47ae-439c-aa53-cf787b767e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:16.005152 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-398271a1-47ae-439c-aa53-cf787b767e08 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:16.006594 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-398271a1-47ae-439c-aa53-cf787b767e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:16.019489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1241/2490] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:16.020394 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600b83e-1618-420a-8efa-76493b5f7e5d None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:16.034284 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600b83e-1618-420a-8efa-76493b5f7e5d None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:16.042667 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2157a65-7d35-4abc-aeb9-56ffe4468d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:16.043532 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2157a65-7d35-4abc-aeb9-56ffe4468d0c admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:16.043976 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2157a65-7d35-4abc-aeb9-56ffe4468d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:16.044439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2157a65-7d35-4abc-aeb9-56ffe4468d0c admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:16.045793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2157a65-7d35-4abc-aeb9-56ffe4468d0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:58:16.046264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2157a65-7d35-4abc-aeb9-56ffe4468d0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:16.048284 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2157a65-7d35-4abc-aeb9-56ffe4468d0c admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:16.068819 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1250/2491] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:58:16.068947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398271a1-47ae-439c-aa53-cf787b767e08 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:16.068947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-398271a1-47ae-439c-aa53-cf787b767e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:16.069521 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2157a65-7d35-4abc-aeb9-56ffe4468d0c admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:16.082636 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2157a65-7d35-4abc-aeb9-56ffe4468d0c admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:16.089144 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09440f-b24a-4517-b023-395763f76d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:16.089834 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09440f-b24a-4517-b023-395763f76d24 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:16.090183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09440f-b24a-4517-b023-395763f76d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:16.090524 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09440f-b24a-4517-b023-395763f76d24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:16.091276 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09440f-b24a-4517-b023-395763f76d24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:16.093294 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09440f-b24a-4517-b023-395763f76d24 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:16.157370 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398271a1-47ae-439c-aa53-cf787b767e08 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:16.162203 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1251/2492] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:58:16 2022] POST /identity/v3/projects => generated 371 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:58:16.167351 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09440f-b24a-4517-b023-395763f76d24 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:16.167534 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7d727-6c58-42de-920f-fece7c72ee41 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdacdcb2451742938164fdd44cdbaebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:16.168142 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7d727-6c58-42de-920f-fece7c72ee41 service placement] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:16.168422 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7d727-6c58-42de-920f-fece7c72ee41 service placement] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:16.168694 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7d727-6c58-42de-920f-fece7c72ee41 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:16.170102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-ab09440f-b24a-4517-b023-395763f76d24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:16.170419 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b2e7d727-6c58-42de-920f-fece7c72ee41 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:16.182994 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e7d727-6c58-42de-920f-fece7c72ee41 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:16.184364 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e7d727-6c58-42de-920f-fece7c72ee41 service placement] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:16.199188 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1242/2493] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:58:16.202726 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7d727-6c58-42de-920f-fece7c72ee41 service placement] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:16.215414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7d727-6c58-42de-920f-fece7c72ee41 service placement] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:16.259189 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:16.292921 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09440f-b24a-4517-b023-395763f76d24 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:16.295294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e76ea7-8f7d-4040-94a6-7755a83992a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:16.296327 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e76ea7-8f7d-4040-94a6-7755a83992a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:16.296327 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e76ea7-8f7d-4040-94a6-7755a83992a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:16.296563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e76ea7-8f7d-4040-94a6-7755a83992a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:16.297078 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e76ea7-8f7d-4040-94a6-7755a83992a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:16.298367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2041adcc-0577-4f37-b6bf-dbbd3488547e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:16.298990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e76ea7-8f7d-4040-94a6-7755a83992a6 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:16.299170 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041adcc-0577-4f37-b6bf-dbbd3488547e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:16.299499 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041adcc-0577-4f37-b6bf-dbbd3488547e admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:16.299949 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041adcc-0577-4f37-b6bf-dbbd3488547e admin admin] PATH_INFO: `/v3/users/3ff39cd0e766400f9d25e3ad465c4faf` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:16.317986 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2041adcc-0577-4f37-b6bf-dbbd3488547e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ff39cd0e766400f9d25e3ad465c4faf)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:16.320532 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2041adcc-0577-4f37-b6bf-dbbd3488547e admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:16.326760 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1243/2494] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:58:16 2022] POST /identity/v3/users => generated 286 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:58:16.343348 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e76ea7-8f7d-4040-94a6-7755a83992a6 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:16.355586 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e76ea7-8f7d-4040-94a6-7755a83992a6 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:16.360536 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f7898-7451-4612-98be-ea868a09a8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:16.361246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f7898-7451-4612-98be-ea868a09a8e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:16.361584 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f7898-7451-4612-98be-ea868a09a8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:16.361931 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f7898-7451-4612-98be-ea868a09a8e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:16.362641 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f7898-7451-4612-98be-ea868a09a8e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:16.364345 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f7898-7451-4612-98be-ea868a09a8e1 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:16.369947 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:16.373470 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-2041adcc-0577-4f37-b6bf-dbbd3488547e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ff39cd0e766400f9d25e3ad465c4faf'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:16.379086 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1244/2495] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:58:16.380428 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f7898-7451-4612-98be-ea868a09a8e1 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:16.384018 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-2041adcc-0577-4f37-b6bf-dbbd3488547e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ff39cd0e766400f9d25e3ad465c4faf'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:16.391255 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f7898-7451-4612-98be-ea868a09a8e1 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:16.392066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-2041adcc-0577-4f37-b6bf-dbbd3488547e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ff39cd0e766400f9d25e3ad465c4faf'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:16.392758 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-2041adcc-0577-4f37-b6bf-dbbd3488547e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ff39cd0e766400f9d25e3ad465c4faf'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:16.393384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-2041adcc-0577-4f37-b6bf-dbbd3488547e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ff39cd0e766400f9d25e3ad465c4faf'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:16.395869 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae62df2-d83e-44f1-b065-13eac9ec75b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:16.396450 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae62df2-d83e-44f1-b065-13eac9ec75b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:16.396728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae62df2-d83e-44f1-b065-13eac9ec75b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:16.396998 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae62df2-d83e-44f1-b065-13eac9ec75b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:16.397622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae62df2-d83e-44f1-b065-13eac9ec75b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:16.399848 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae62df2-d83e-44f1-b065-13eac9ec75b0 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:16.410628 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1245/2496] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:58:16.414186 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1252/2497] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:58:16 2022] DELETE /identity/v3/users/3ff39cd0e766400f9d25e3ad465c4faf => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:16.415023 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae62df2-d83e-44f1-b065-13eac9ec75b0 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:16.416785 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-9ae62df2-d83e-44f1-b065-13eac9ec75b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:16.430109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae62df2-d83e-44f1-b065-13eac9ec75b0 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:16.489207 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d856bf6-e01e-4a25-9bb8-9106a0cb888c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:16.490018 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d856bf6-e01e-4a25-9bb8-9106a0cb888c admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:16.490349 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d856bf6-e01e-4a25-9bb8-9106a0cb888c admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:16.490626 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d856bf6-e01e-4a25-9bb8-9106a0cb888c admin admin] PATH_INFO: `/v3/projects/e5fdc920045d4fabac1e51adb944ff3f/users/d69d5ca56a8e43ca9d91920684f2dac8/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:16.507575 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d856bf6-e01e-4a25-9bb8-9106a0cb888c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5fdc920045d4fabac1e51adb944ff3f, user_id=d69d5ca56a8e43ca9d91920684f2dac8, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:16.510344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d856bf6-e01e-4a25-9bb8-9106a0cb888c admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:16.527818 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1246/2498] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:58:16 2022] PUT /identity/v3/projects/e5fdc920045d4fabac1e51adb944ff3f/users/d69d5ca56a8e43ca9d91920684f2dac8/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:58:16.533740 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d980c6-1b51-42f2-86b1-0d02b3341648 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:16.534115 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d980c6-1b51-42f2-86b1-0d02b3341648 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:16.534451 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d980c6-1b51-42f2-86b1-0d02b3341648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:16.555554 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-32d980c6-1b51-42f2-86b1-0d02b3341648 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:16.556763 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-32d980c6-1b51-42f2-86b1-0d02b3341648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:16.605251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d856bf6-e01e-4a25-9bb8-9106a0cb888c admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:16.607510 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-4d856bf6-e01e-4a25-9bb8-9106a0cb888c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:16.629014 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1253/2499] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:16.640599 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d980c6-1b51-42f2-86b1-0d02b3341648 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:16.642439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-32d980c6-1b51-42f2-86b1-0d02b3341648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:16.682785 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d856bf6-e01e-4a25-9bb8-9106a0cb888c admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:16.687739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666833b-7bdd-4e46-b0b5-5e87771691c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:16.688377 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666833b-7bdd-4e46-b0b5-5e87771691c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:16.688684 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666833b-7bdd-4e46-b0b5-5e87771691c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:16.688955 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666833b-7bdd-4e46-b0b5-5e87771691c3 admin admin] PATH_INFO: `/v3/projects/6a4d75f50b854a909607941db6debb87` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:16.690089 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666833b-7bdd-4e46-b0b5-5e87771691c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a4d75f50b854a909607941db6debb87)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:16.693061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666833b-7bdd-4e46-b0b5-5e87771691c3 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:16.706933 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d980c6-1b51-42f2-86b1-0d02b3341648 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:16.711572 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb264dd-b5ce-4afa-ad0e-01a061508bd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:16.712466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb264dd-b5ce-4afa-ad0e-01a061508bd7 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:16.712956 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb264dd-b5ce-4afa-ad0e-01a061508bd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:16.713234 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb264dd-b5ce-4afa-ad0e-01a061508bd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:16.721798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:16.723796 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb264dd-b5ce-4afa-ad0e-01a061508bd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:16.725450 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb264dd-b5ce-4afa-ad0e-01a061508bd7 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:16.736495 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1254/2500] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:58:16.760996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-0666833b-7bdd-4e46-b0b5-5e87771691c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a4d75f50b854a909607941db6debb87', 'request_id': 'req-0666833b-7bdd-4e46-b0b5-5e87771691c3'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:16.768755 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-0666833b-7bdd-4e46-b0b5-5e87771691c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a4d75f50b854a909607941db6debb87', 'request_id': 'req-0666833b-7bdd-4e46-b0b5-5e87771691c3'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:16.775201 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-0666833b-7bdd-4e46-b0b5-5e87771691c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a4d75f50b854a909607941db6debb87', 'request_id': 'req-0666833b-7bdd-4e46-b0b5-5e87771691c3'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:16.786639 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-0666833b-7bdd-4e46-b0b5-5e87771691c3 admin admin] The token cache is being invalidate because project 6a4d75f50b854a909607941db6debb87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:58:16.787259 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-0666833b-7bdd-4e46-b0b5-5e87771691c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:16.787840 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-0666833b-7bdd-4e46-b0b5-5e87771691c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:16.790011 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1247/2501] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:58:16 2022] DELETE /identity/v3/projects/6a4d75f50b854a909607941db6debb87 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:58:16.795993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa614073-0f3b-4a16-87b4-eed23cfd3e3e None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:16.796567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa614073-0f3b-4a16-87b4-eed23cfd3e3e None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:16.796567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa614073-0f3b-4a16-87b4-eed23cfd3e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:16.818755 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-fa614073-0f3b-4a16-87b4-eed23cfd3e3e None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:16.820714 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-fa614073-0f3b-4a16-87b4-eed23cfd3e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:16.890409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1255/2502] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:16.899107 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc89375-a787-4b9c-8055-d33bf54aa8a9 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:16.899564 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc89375-a787-4b9c-8055-d33bf54aa8a9 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:16.899916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc89375-a787-4b9c-8055-d33bf54aa8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:16.923770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-dcc89375-a787-4b9c-8055-d33bf54aa8a9 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:16.925122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-dcc89375-a787-4b9c-8055-d33bf54aa8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:16.934261 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1248/2503] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:16.938492 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa614073-0f3b-4a16-87b4-eed23cfd3e3e None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:16.944538 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:16.949237 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa614073-0f3b-4a16-87b4-eed23cfd3e3e None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:16.955741 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d8d55-ac11-41aa-9e57-a647416c9e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:16.956526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d8d55-ac11-41aa-9e57-a647416c9e77 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:16.957065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d8d55-ac11-41aa-9e57-a647416c9e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:16.957484 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d8d55-ac11-41aa-9e57-a647416c9e77 admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:16.958264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d8d55-ac11-41aa-9e57-a647416c9e77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:58:16.958668 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d8d55-ac11-41aa-9e57-a647416c9e77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:16.960370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d8d55-ac11-41aa-9e57-a647416c9e77 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:16.980096 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1256/2504] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:58:16.984767 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc89375-a787-4b9c-8055-d33bf54aa8a9 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:16.996935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc89375-a787-4b9c-8055-d33bf54aa8a9 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:17.001685 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635894c2-0ca2-4d35-940f-f84c27719989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:17.002290 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635894c2-0ca2-4d35-940f-f84c27719989 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:17.002566 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635894c2-0ca2-4d35-940f-f84c27719989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:17.002833 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635894c2-0ca2-4d35-940f-f84c27719989 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:17.003583 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635894c2-0ca2-4d35-940f-f84c27719989 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:17.005568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635894c2-0ca2-4d35-940f-f84c27719989 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:17.036601 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1249/2505] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:58:16 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:58:17.041736 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d8d55-ac11-41aa-9e57-a647416c9e77 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:17.054360 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d8d55-ac11-41aa-9e57-a647416c9e77 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:17.120231 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fecad55-3763-48fc-9976-2fb9de1d70c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:17.121378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fecad55-3763-48fc-9976-2fb9de1d70c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:17.121378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fecad55-3763-48fc-9976-2fb9de1d70c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:17.121707 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fecad55-3763-48fc-9976-2fb9de1d70c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:17.122205 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fecad55-3763-48fc-9976-2fb9de1d70c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:17.124209 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fecad55-3763-48fc-9976-2fb9de1d70c6 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:17.154790 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1257/2506] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:58:17 2022] POST /identity/v3/users => generated 285 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:58:17.156150 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635894c2-0ca2-4d35-940f-f84c27719989 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:17.170067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635894c2-0ca2-4d35-940f-f84c27719989 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:17.175539 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab9dc7-f2c0-4e35-a2c7-a4425e7b00d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:17.176239 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab9dc7-f2c0-4e35-a2c7-a4425e7b00d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:17.176588 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab9dc7-f2c0-4e35-a2c7-a4425e7b00d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:17.177732 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab9dc7-f2c0-4e35-a2c7-a4425e7b00d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:17.179021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebab9dc7-f2c0-4e35-a2c7-a4425e7b00d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:17.182374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebab9dc7-f2c0-4e35-a2c7-a4425e7b00d8 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:17.195825 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1250/2507] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:58:17.200525 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fecad55-3763-48fc-9976-2fb9de1d70c6 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:17.214621 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fecad55-3763-48fc-9976-2fb9de1d70c6 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:17.221840 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce05d40-36dd-4f87-b369-435611cd9ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:17.222824 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce05d40-36dd-4f87-b369-435611cd9ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:17.223130 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce05d40-36dd-4f87-b369-435611cd9ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:17.223398 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce05d40-36dd-4f87-b369-435611cd9ee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:17.224047 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce05d40-36dd-4f87-b369-435611cd9ee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:17.225669 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce05d40-36dd-4f87-b369-435611cd9ee5 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:17.241462 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1258/2508] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:58:17.246245 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab9dc7-f2c0-4e35-a2c7-a4425e7b00d8 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:17.256526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab9dc7-f2c0-4e35-a2c7-a4425e7b00d8 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:17.262228 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca35953-cf59-4c7b-b04c-29aa3b17b29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:17.263117 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca35953-cf59-4c7b-b04c-29aa3b17b29a admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:17.263472 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca35953-cf59-4c7b-b04c-29aa3b17b29a admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:17.263610 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca35953-cf59-4c7b-b04c-29aa3b17b29a admin admin] PATH_INFO: `/v3/projects/d18cc726e9154d3e8fc3187949f14fd6/users/6a319573c2624d19b559e59bd05a689b/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:17.280992 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca35953-cf59-4c7b-b04c-29aa3b17b29a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d18cc726e9154d3e8fc3187949f14fd6, user_id=6a319573c2624d19b559e59bd05a689b, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:17.284391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca35953-cf59-4c7b-b04c-29aa3b17b29a admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:17.298591 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1251/2509] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:58:17 2022] PUT /identity/v3/projects/d18cc726e9154d3e8fc3187949f14fd6/users/6a319573c2624d19b559e59bd05a689b/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:58:17.310921 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f081b8-370a-466f-b7f5-49c75e46b89f None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:17.311197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f081b8-370a-466f-b7f5-49c75e46b89f None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:17.311440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f081b8-370a-466f-b7f5-49c75e46b89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:17.344260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-f7f081b8-370a-466f-b7f5-49c75e46b89f None None] MFA Rules not processed for user `6a319573c2624d19b559e59bd05a689b`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:17.346326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f7f081b8-370a-466f-b7f5-49c75e46b89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:17.451142 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1259/2510] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:17.456138 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca35953-cf59-4c7b-b04c-29aa3b17b29a admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:17.588634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca35953-cf59-4c7b-b04c-29aa3b17b29a admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:17.593592 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28822a-436b-40d0-92df-28c4d15a9c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:17.594200 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28822a-436b-40d0-92df-28c4d15a9c83 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:17.594805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28822a-436b-40d0-92df-28c4d15a9c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:17.595074 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28822a-436b-40d0-92df-28c4d15a9c83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:17.596320 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f28822a-436b-40d0-92df-28c4d15a9c83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:17.597732 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f28822a-436b-40d0-92df-28c4d15a9c83 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:17.630673 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1252/2511] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:58:17 2022] POST /identity/v3/projects => generated 369 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:58:17.636106 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f081b8-370a-466f-b7f5-49c75e46b89f None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:17.647587 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f081b8-370a-466f-b7f5-49c75e46b89f None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:17.718364 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a8f1d-9112-43f3-b614-fa3e6e6382d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:17.719116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a8f1d-9112-43f3-b614-fa3e6e6382d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:17.719727 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a8f1d-9112-43f3-b614-fa3e6e6382d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:17.720078 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a8f1d-9112-43f3-b614-fa3e6e6382d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:17.720781 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a8f1d-9112-43f3-b614-fa3e6e6382d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:17.722975 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a8f1d-9112-43f3-b614-fa3e6e6382d0 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:17.732749 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:17.749801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1260/2512] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:58:17 2022] POST /identity/v3/users => generated 291 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:58:17.753655 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28822a-436b-40d0-92df-28c4d15a9c83 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:17.768077 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28822a-436b-40d0-92df-28c4d15a9c83 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:17.776047 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffbe7e-e096-4785-816c-cdda63e2ee4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:17.776967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffbe7e-e096-4785-816c-cdda63e2ee4c admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:17.777369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffbe7e-e096-4785-816c-cdda63e2ee4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:17.777733 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffbe7e-e096-4785-816c-cdda63e2ee4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:17.778943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffbe7e-e096-4785-816c-cdda63e2ee4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:17.781447 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffbe7e-e096-4785-816c-cdda63e2ee4c admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:17.816908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1253/2513] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:58:17.822253 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a8f1d-9112-43f3-b614-fa3e6e6382d0 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:17.833050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a8f1d-9112-43f3-b614-fa3e6e6382d0 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:17.838465 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c34cf-7667-4e8d-962e-c057a7e1057d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:17.839063 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c34cf-7667-4e8d-962e-c057a7e1057d admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:17.839300 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c34cf-7667-4e8d-962e-c057a7e1057d admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:17.839568 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c34cf-7667-4e8d-962e-c057a7e1057d admin admin] PATH_INFO: `/v3/projects/0f0ee6f64acd4064bf1cb1009f069bcb/users/fccb76cfc71d40e79f25eda860f62cd9/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:17.881428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c34cf-7667-4e8d-962e-c057a7e1057d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f0ee6f64acd4064bf1cb1009f069bcb, user_id=fccb76cfc71d40e79f25eda860f62cd9, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:17.888369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c34cf-7667-4e8d-962e-c057a7e1057d admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:17.921015 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1261/2514] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:58:17 2022] PUT /identity/v3/projects/0f0ee6f64acd4064bf1cb1009f069bcb/users/fccb76cfc71d40e79f25eda860f62cd9/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:17.921373 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffbe7e-e096-4785-816c-cdda63e2ee4c admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:17.933760 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffbe7e-e096-4785-816c-cdda63e2ee4c admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:17.964341 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:18.049677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2776e6-8c1d-4d44-9402-f733e15a9e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:18.050331 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2776e6-8c1d-4d44-9402-f733e15a9e36 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:18.050617 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2776e6-8c1d-4d44-9402-f733e15a9e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:18.050892 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2776e6-8c1d-4d44-9402-f733e15a9e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:18.051577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2776e6-8c1d-4d44-9402-f733e15a9e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:18.053357 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2776e6-8c1d-4d44-9402-f733e15a9e36 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:18.068572 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1254/2515] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:58:18.069875 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c34cf-7667-4e8d-962e-c057a7e1057d admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:18.096466 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c34cf-7667-4e8d-962e-c057a7e1057d admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:18.103102 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe882743-ddd4-41ea-8d5a-3fd4b714b688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:18.103835 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe882743-ddd4-41ea-8d5a-3fd4b714b688 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:18.104090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe882743-ddd4-41ea-8d5a-3fd4b714b688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:18.104430 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe882743-ddd4-41ea-8d5a-3fd4b714b688 admin admin] PATH_INFO: `/v3/projects/0f0ee6f64acd4064bf1cb1009f069bcb/users/fccb76cfc71d40e79f25eda860f62cd9/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:18.106787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe882743-ddd4-41ea-8d5a-3fd4b714b688 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f0ee6f64acd4064bf1cb1009f069bcb, user_id=fccb76cfc71d40e79f25eda860f62cd9, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:18.110238 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe882743-ddd4-41ea-8d5a-3fd4b714b688 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:18.136262 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1262/2516] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:58:18 2022] PUT /identity/v3/projects/0f0ee6f64acd4064bf1cb1009f069bcb/users/fccb76cfc71d40e79f25eda860f62cd9/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:18.137500 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac3886-e873-48a7-8e0c-692111668640 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:18.137870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac3886-e873-48a7-8e0c-692111668640 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:18.138092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac3886-e873-48a7-8e0c-692111668640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:18.176659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-1dac3886-e873-48a7-8e0c-692111668640 None None] MFA Rules not processed for user `fccb76cfc71d40e79f25eda860f62cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:18.179248 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-1dac3886-e873-48a7-8e0c-692111668640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:18.296399 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1255/2517] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:18 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:18.310784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe882743-ddd4-41ea-8d5a-3fd4b714b688 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:18.313548 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-fe882743-ddd4-41ea-8d5a-3fd4b714b688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:18.408585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe882743-ddd4-41ea-8d5a-3fd4b714b688 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:18.419510 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2b853-025f-4600-a9cd-073466609b11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:18.419510 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2b853-025f-4600-a9cd-073466609b11 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:18.419510 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2b853-025f-4600-a9cd-073466609b11 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:18.419510 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2b853-025f-4600-a9cd-073466609b11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:18.432863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e2b853-025f-4600-a9cd-073466609b11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:18.434402 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e2b853-025f-4600-a9cd-073466609b11 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:18.454372 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1263/2518] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:18 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:58:18.521770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac3886-e873-48a7-8e0c-692111668640 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:18.521770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-1dac3886-e873-48a7-8e0c-692111668640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:18.540169 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac3886-e873-48a7-8e0c-692111668640 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:18.548358 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1ea70-b9a7-4463-83d4-ae006bc82bb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:18.549176 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1ea70-b9a7-4463-83d4-ae006bc82bb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:18.549465 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1ea70-b9a7-4463-83d4-ae006bc82bb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:18.549738 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1ea70-b9a7-4463-83d4-ae006bc82bb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:18.646396 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1ea70-b9a7-4463-83d4-ae006bc82bb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:18.649306 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1ea70-b9a7-4463-83d4-ae006bc82bb7 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:18.659649 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1256/2519] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:58:21.381209 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc69def-5164-4517-9fb4-fd3b307f10bf None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:21.381693 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc69def-5164-4517-9fb4-fd3b307f10bf None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:21.381806 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc69def-5164-4517-9fb4-fd3b307f10bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:21.424367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-0cc69def-5164-4517-9fb4-fd3b307f10bf None None] MFA Rules not processed for user `d69d5ca56a8e43ca9d91920684f2dac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:21.425590 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-0cc69def-5164-4517-9fb4-fd3b307f10bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:21.559393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1264/2520] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:21.563951 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1ea70-b9a7-4463-83d4-ae006bc82bb7 service cinder] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:21.565574 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-9fa1ea70-b9a7-4463-83d4-ae006bc82bb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:21.629879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:21.652467 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1ea70-b9a7-4463-83d4-ae006bc82bb7 service cinder] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:21.657648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf019f4-2dac-4eae-a1c8-89cdbe1397b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:21.658271 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf019f4-2dac-4eae-a1c8-89cdbe1397b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:21.658549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf019f4-2dac-4eae-a1c8-89cdbe1397b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:21.658820 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf019f4-2dac-4eae-a1c8-89cdbe1397b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:21.659507 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf019f4-2dac-4eae-a1c8-89cdbe1397b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:21.661432 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf019f4-2dac-4eae-a1c8-89cdbe1397b8 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:21.692952 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1257/2521] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:58:21 2022] POST /identity/v3/projects => generated 371 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:58:21.694233 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc69def-5164-4517-9fb4-fd3b307f10bf None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:21.706842 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc69def-5164-4517-9fb4-fd3b307f10bf None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:21.760170 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:21.779418 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f99be3-a97d-406d-8616-bf320cb5fb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:21.780168 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f99be3-a97d-406d-8616-bf320cb5fb19 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:21.780552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f99be3-a97d-406d-8616-bf320cb5fb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:21.780927 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f99be3-a97d-406d-8616-bf320cb5fb19 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:21.781640 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f99be3-a97d-406d-8616-bf320cb5fb19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:21.783816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f99be3-a97d-406d-8616-bf320cb5fb19 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:21.826477 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1265/2522] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:58:21 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:58:21.830312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf019f4-2dac-4eae-a1c8-89cdbe1397b8 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:21.842631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf019f4-2dac-4eae-a1c8-89cdbe1397b8 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:21.857629 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee4daee-c04f-4bf7-9c04-677b38ae0568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:21.858326 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee4daee-c04f-4bf7-9c04-677b38ae0568 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:21.858912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee4daee-c04f-4bf7-9c04-677b38ae0568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:21.862020 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee4daee-c04f-4bf7-9c04-677b38ae0568 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:21.862772 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee4daee-c04f-4bf7-9c04-677b38ae0568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:21.868565 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee4daee-c04f-4bf7-9c04-677b38ae0568 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:21.892350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1258/2523] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:21 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:58:21.896542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f99be3-a97d-406d-8616-bf320cb5fb19 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:21.929771 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f99be3-a97d-406d-8616-bf320cb5fb19 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:21.947969 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ddd1e-2896-4e9d-ab39-0eb5bb26b8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:21.948785 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ddd1e-2896-4e9d-ab39-0eb5bb26b8c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:21.949068 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ddd1e-2896-4e9d-ab39-0eb5bb26b8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:21.949338 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ddd1e-2896-4e9d-ab39-0eb5bb26b8c6 admin admin] PATH_INFO: `/v3/projects/4a00d4bba4a2464f8880c7b4de32bd54/users/5f99a7b91cd54ba7a9bd79484a7fb436/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:21.972797 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ddd1e-2896-4e9d-ab39-0eb5bb26b8c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a00d4bba4a2464f8880c7b4de32bd54, user_id=5f99a7b91cd54ba7a9bd79484a7fb436, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:21.979882 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ddd1e-2896-4e9d-ab39-0eb5bb26b8c6 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:22.023812 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1266/2524] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:58:21 2022] PUT /identity/v3/projects/4a00d4bba4a2464f8880c7b4de32bd54/users/5f99a7b91cd54ba7a9bd79484a7fb436/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:22.028251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee4daee-c04f-4bf7-9c04-677b38ae0568 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:22.043833 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee4daee-c04f-4bf7-9c04-677b38ae0568 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:22.192824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b519b81-c20e-4491-ac0e-3912a360e5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:22.193766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b519b81-c20e-4491-ac0e-3912a360e5ca admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:22.194171 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b519b81-c20e-4491-ac0e-3912a360e5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:22.194435 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b519b81-c20e-4491-ac0e-3912a360e5ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:22.195313 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b519b81-c20e-4491-ac0e-3912a360e5ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:22.197268 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b519b81-c20e-4491-ac0e-3912a360e5ca admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:22.218676 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1259/2525] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:58:22.220770 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ddd1e-2896-4e9d-ab39-0eb5bb26b8c6 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:22.235553 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ddd1e-2896-4e9d-ab39-0eb5bb26b8c6 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:22.241609 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc06dd-a7ef-416c-822b-d811f08ada99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:22.243114 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc06dd-a7ef-416c-822b-d811f08ada99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:22.243713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc06dd-a7ef-416c-822b-d811f08ada99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:22.243860 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc06dd-a7ef-416c-822b-d811f08ada99 admin admin] PATH_INFO: `/v3/projects/4a00d4bba4a2464f8880c7b4de32bd54/users/5f99a7b91cd54ba7a9bd79484a7fb436/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:22.246886 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc06dd-a7ef-416c-822b-d811f08ada99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a00d4bba4a2464f8880c7b4de32bd54, user_id=5f99a7b91cd54ba7a9bd79484a7fb436, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:22.251586 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc06dd-a7ef-416c-822b-d811f08ada99 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:22.269817 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1267/2526] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:58:22 2022] PUT /identity/v3/projects/4a00d4bba4a2464f8880c7b4de32bd54/users/5f99a7b91cd54ba7a9bd79484a7fb436/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:22.649786 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b519b81-c20e-4491-ac0e-3912a360e5ca admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:22.661008 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b519b81-c20e-4491-ac0e-3912a360e5ca admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:22.759213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103fae0-5b0a-4f18-8dfa-9633b58bb627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:22.759982 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103fae0-5b0a-4f18-8dfa-9633b58bb627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:22.760333 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103fae0-5b0a-4f18-8dfa-9633b58bb627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:22.760680 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103fae0-5b0a-4f18-8dfa-9633b58bb627 admin admin] PATH_INFO: `/v3/users/6a319573c2624d19b559e59bd05a689b` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:22.762486 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103fae0-5b0a-4f18-8dfa-9633b58bb627 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a319573c2624d19b559e59bd05a689b)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:22.764650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103fae0-5b0a-4f18-8dfa-9633b58bb627 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:22.827573 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-8103fae0-5b0a-4f18-8dfa-9633b58bb627 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a319573c2624d19b559e59bd05a689b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:22.838131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:22.843622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-8103fae0-5b0a-4f18-8dfa-9633b58bb627 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a319573c2624d19b559e59bd05a689b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:22.856056 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-8103fae0-5b0a-4f18-8dfa-9633b58bb627 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a319573c2624d19b559e59bd05a689b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:22.868259 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-8103fae0-5b0a-4f18-8dfa-9633b58bb627 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a319573c2624d19b559e59bd05a689b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:22.869344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-8103fae0-5b0a-4f18-8dfa-9633b58bb627 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a319573c2624d19b559e59bd05a689b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:22.876622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1260/2527] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:58:22 2022] DELETE /identity/v3/users/6a319573c2624d19b559e59bd05a689b => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:22.881353 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85eaaf-9268-4fab-912d-f4dc8900b6d3 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:22.881802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85eaaf-9268-4fab-912d-f4dc8900b6d3 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:22.882123 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85eaaf-9268-4fab-912d-f4dc8900b6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:22.903964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-5b85eaaf-9268-4fab-912d-f4dc8900b6d3 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:22.905261 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-5b85eaaf-9268-4fab-912d-f4dc8900b6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:22.976914 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1268/2528] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:22.986743 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103fae0-5b0a-4f18-8dfa-9633b58bb627 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:22.988460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-8103fae0-5b0a-4f18-8dfa-9633b58bb627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:23.053842 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103fae0-5b0a-4f18-8dfa-9633b58bb627 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:23.058480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa1d14-e030-4032-95b8-2ed18fdde185 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:23.059197 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa1d14-e030-4032-95b8-2ed18fdde185 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:23.060155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa1d14-e030-4032-95b8-2ed18fdde185 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:23.060433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa1d14-e030-4032-95b8-2ed18fdde185 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:23.071593 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faa1d14-e030-4032-95b8-2ed18fdde185 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:23.073100 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faa1d14-e030-4032-95b8-2ed18fdde185 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:23.083740 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1261/2529] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:58:23.441525 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85eaaf-9268-4fab-912d-f4dc8900b6d3 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:23.443324 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-5b85eaaf-9268-4fab-912d-f4dc8900b6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:23.467763 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85eaaf-9268-4fab-912d-f4dc8900b6d3 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:23.483100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5954b-f0ff-4ba4-b949-3da02a12d1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:23.483905 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5954b-f0ff-4ba4-b949-3da02a12d1e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:23.484190 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5954b-f0ff-4ba4-b949-3da02a12d1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:23.484464 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5954b-f0ff-4ba4-b949-3da02a12d1e4 admin admin] PATH_INFO: `/v3/projects/d18cc726e9154d3e8fc3187949f14fd6` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:23.486058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be5954b-f0ff-4ba4-b949-3da02a12d1e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d18cc726e9154d3e8fc3187949f14fd6)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:23.488058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be5954b-f0ff-4ba4-b949-3da02a12d1e4 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:23.556674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3be5954b-f0ff-4ba4-b949-3da02a12d1e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd18cc726e9154d3e8fc3187949f14fd6', 'request_id': 'req-3be5954b-f0ff-4ba4-b949-3da02a12d1e4'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:23.568107 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3be5954b-f0ff-4ba4-b949-3da02a12d1e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd18cc726e9154d3e8fc3187949f14fd6', 'request_id': 'req-3be5954b-f0ff-4ba4-b949-3da02a12d1e4'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:23.577596 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3be5954b-f0ff-4ba4-b949-3da02a12d1e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd18cc726e9154d3e8fc3187949f14fd6', 'request_id': 'req-3be5954b-f0ff-4ba4-b949-3da02a12d1e4'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:23.605398 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3be5954b-f0ff-4ba4-b949-3da02a12d1e4 admin admin] The token cache is being invalidate because project d18cc726e9154d3e8fc3187949f14fd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:58:23.605398 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3be5954b-f0ff-4ba4-b949-3da02a12d1e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:23.605398 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3be5954b-f0ff-4ba4-b949-3da02a12d1e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:23.607325 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1269/2530] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:58:23 2022] DELETE /identity/v3/projects/d18cc726e9154d3e8fc3187949f14fd6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:23.611061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa1d14-e030-4032-95b8-2ed18fdde185 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:23.623871 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa1d14-e030-4032-95b8-2ed18fdde185 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:23.708638 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551e807-6c14-4cd6-8f01-f11c63173c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:23.709363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551e807-6c14-4cd6-8f01-f11c63173c94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:23.709718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551e807-6c14-4cd6-8f01-f11c63173c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:23.710060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551e807-6c14-4cd6-8f01-f11c63173c94 admin admin] PATH_INFO: `/v3/users/fccb76cfc71d40e79f25eda860f62cd9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:23.712464 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a551e807-6c14-4cd6-8f01-f11c63173c94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fccb76cfc71d40e79f25eda860f62cd9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:23.714515 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a551e807-6c14-4cd6-8f01-f11c63173c94 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:23.776749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-a551e807-6c14-4cd6-8f01-f11c63173c94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fccb76cfc71d40e79f25eda860f62cd9'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:23.791362 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-a551e807-6c14-4cd6-8f01-f11c63173c94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fccb76cfc71d40e79f25eda860f62cd9'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:23.799156 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-a551e807-6c14-4cd6-8f01-f11c63173c94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fccb76cfc71d40e79f25eda860f62cd9'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:23.802431 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:23.822626 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-a551e807-6c14-4cd6-8f01-f11c63173c94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fccb76cfc71d40e79f25eda860f62cd9'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:23.823743 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-a551e807-6c14-4cd6-8f01-f11c63173c94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fccb76cfc71d40e79f25eda860f62cd9'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:23.848954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1262/2531] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:58:23 2022] DELETE /identity/v3/users/fccb76cfc71d40e79f25eda860f62cd9 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:58:24.461560 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5954b-f0ff-4ba4-b949-3da02a12d1e4 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:24.463120 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-3be5954b-f0ff-4ba4-b949-3da02a12d1e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:24.485914 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5954b-f0ff-4ba4-b949-3da02a12d1e4 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:24.528177 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:24.616644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae2503-7e5c-4e0d-921a-f57826774146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:24.617341 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae2503-7e5c-4e0d-921a-f57826774146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:24.617605 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae2503-7e5c-4e0d-921a-f57826774146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:24.617877 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae2503-7e5c-4e0d-921a-f57826774146 admin admin] PATH_INFO: `/v3/projects/0f0ee6f64acd4064bf1cb1009f069bcb` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:24.620577 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae2503-7e5c-4e0d-921a-f57826774146 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f0ee6f64acd4064bf1cb1009f069bcb)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:24.622551 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae2503-7e5c-4e0d-921a-f57826774146 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:24.739604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-eeae2503-7e5c-4e0d-921a-f57826774146 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f0ee6f64acd4064bf1cb1009f069bcb', 'request_id': 'req-eeae2503-7e5c-4e0d-921a-f57826774146'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:24.756146 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-eeae2503-7e5c-4e0d-921a-f57826774146 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f0ee6f64acd4064bf1cb1009f069bcb', 'request_id': 'req-eeae2503-7e5c-4e0d-921a-f57826774146'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:24.775589 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-eeae2503-7e5c-4e0d-921a-f57826774146 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f0ee6f64acd4064bf1cb1009f069bcb', 'request_id': 'req-eeae2503-7e5c-4e0d-921a-f57826774146'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:24.822806 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-eeae2503-7e5c-4e0d-921a-f57826774146 admin admin] The token cache is being invalidate because project 0f0ee6f64acd4064bf1cb1009f069bcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:58:24.825764 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-eeae2503-7e5c-4e0d-921a-f57826774146 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:24.827396 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-eeae2503-7e5c-4e0d-921a-f57826774146 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:24.831152 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1270/2532] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:58:24 2022] DELETE /identity/v3/projects/0f0ee6f64acd4064bf1cb1009f069bcb => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:24.835967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b5d0d-1c93-431b-8aa3-1ae92e7ae686 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:24.836521 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b5d0d-1c93-431b-8aa3-1ae92e7ae686 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:24.836521 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b5d0d-1c93-431b-8aa3-1ae92e7ae686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:24.869615 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-493b5d0d-1c93-431b-8aa3-1ae92e7ae686 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:24.873078 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-493b5d0d-1c93-431b-8aa3-1ae92e7ae686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:24.977858 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1263/2533] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:24.985711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238c66f-18a4-450b-99e5-535ed4fe5f7e None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:24.985998 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238c66f-18a4-450b-99e5-535ed4fe5f7e None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:24.986464 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238c66f-18a4-450b-99e5-535ed4fe5f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:25.015912 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-6238c66f-18a4-450b-99e5-535ed4fe5f7e None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:25.017173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-6238c66f-18a4-450b-99e5-535ed4fe5f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:25.028439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1271/2534] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:25.032542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b5d0d-1c93-431b-8aa3-1ae92e7ae686 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:25.049853 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b5d0d-1c93-431b-8aa3-1ae92e7ae686 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:25.061874 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1750c-f8b8-44de-88f5-29f3f6fbc296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:25.062712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1750c-f8b8-44de-88f5-29f3f6fbc296 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:25.063176 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1750c-f8b8-44de-88f5-29f3f6fbc296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:25.063556 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1750c-f8b8-44de-88f5-29f3f6fbc296 admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:25.064350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1750c-f8b8-44de-88f5-29f3f6fbc296 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:58:25.064873 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1750c-f8b8-44de-88f5-29f3f6fbc296 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:25.067435 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1750c-f8b8-44de-88f5-29f3f6fbc296 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:25.085390 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1264/2535] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:58:25.089995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238c66f-18a4-450b-99e5-535ed4fe5f7e None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:25.100124 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238c66f-18a4-450b-99e5-535ed4fe5f7e None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:25.106884 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec2fd9-dde9-435b-a96d-3752cc6e6218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:25.107975 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec2fd9-dde9-435b-a96d-3752cc6e6218 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:25.108468 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec2fd9-dde9-435b-a96d-3752cc6e6218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:25.108934 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec2fd9-dde9-435b-a96d-3752cc6e6218 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:25.110116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec2fd9-dde9-435b-a96d-3752cc6e6218 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:25.112089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec2fd9-dde9-435b-a96d-3752cc6e6218 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:25.148247 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1272/2536] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:58:25 2022] POST /identity/v3/projects => generated 387 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:58:25.152753 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1750c-f8b8-44de-88f5-29f3f6fbc296 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:25.169564 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1750c-f8b8-44de-88f5-29f3f6fbc296 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:25.294589 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee10448-2bc4-4f09-b352-d96c0b5e7992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:25.295245 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee10448-2bc4-4f09-b352-d96c0b5e7992 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:25.295526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee10448-2bc4-4f09-b352-d96c0b5e7992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:25.295798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee10448-2bc4-4f09-b352-d96c0b5e7992 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:25.296447 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee10448-2bc4-4f09-b352-d96c0b5e7992 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:25.298437 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee10448-2bc4-4f09-b352-d96c0b5e7992 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:25.337852 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1265/2537] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:58:25 2022] POST /identity/v3/users => generated 294 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:58:25.338376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec2fd9-dde9-435b-a96d-3752cc6e6218 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:25.352920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec2fd9-dde9-435b-a96d-3752cc6e6218 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:25.358955 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5071e82-7c51-4d3f-9152-a7188ab26a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:25.359817 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5071e82-7c51-4d3f-9152-a7188ab26a5f admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:25.360230 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5071e82-7c51-4d3f-9152-a7188ab26a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:25.360630 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5071e82-7c51-4d3f-9152-a7188ab26a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:25.361711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5071e82-7c51-4d3f-9152-a7188ab26a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:25.363689 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5071e82-7c51-4d3f-9152-a7188ab26a5f admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:25.386424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee10448-2bc4-4f09-b352-d96c0b5e7992 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:25.387700 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1273/2538] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:58:25.399797 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee10448-2bc4-4f09-b352-d96c0b5e7992 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:25.406049 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e7346-147e-4c5a-9850-81f5b30e7e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:25.406793 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e7346-147e-4c5a-9850-81f5b30e7e88 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:25.407219 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e7346-147e-4c5a-9850-81f5b30e7e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:25.407599 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e7346-147e-4c5a-9850-81f5b30e7e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:25.408449 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204e7346-147e-4c5a-9850-81f5b30e7e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:25.410521 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204e7346-147e-4c5a-9850-81f5b30e7e88 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:25.431894 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5071e82-7c51-4d3f-9152-a7188ab26a5f admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:25.439173 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1266/2539] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:58:25.447594 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5071e82-7c51-4d3f-9152-a7188ab26a5f admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:25.453215 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4730c20-8894-4cb6-9fb0-8fdd172e190b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:25.454046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4730c20-8894-4cb6-9fb0-8fdd172e190b admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:25.454431 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4730c20-8894-4cb6-9fb0-8fdd172e190b admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:25.454818 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4730c20-8894-4cb6-9fb0-8fdd172e190b admin admin] PATH_INFO: `/v3/projects/5b58f1c5c2b94ca6b1ea27cb83454378/users/be4920c391b94cb39b1b54ddd7840326/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:25.479595 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4730c20-8894-4cb6-9fb0-8fdd172e190b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b58f1c5c2b94ca6b1ea27cb83454378, user_id=be4920c391b94cb39b1b54ddd7840326, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:25.482942 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4730c20-8894-4cb6-9fb0-8fdd172e190b admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:25.515128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55971377-8e76-494f-9471-c820a4d2d1af None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:25.515478 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55971377-8e76-494f-9471-c820a4d2d1af None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:25.515887 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55971377-8e76-494f-9471-c820a4d2d1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:25.519071 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1274/2540] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:58:25 2022] PUT /identity/v3/projects/5b58f1c5c2b94ca6b1ea27cb83454378/users/be4920c391b94cb39b1b54ddd7840326/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:25.582234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-55971377-8e76-494f-9471-c820a4d2d1af None None] MFA Rules not processed for user `be4920c391b94cb39b1b54ddd7840326`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:25.583990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-55971377-8e76-494f-9471-c820a4d2d1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:25.677517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:25.724271 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1267/2541] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:25.729570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4730c20-8894-4cb6-9fb0-8fdd172e190b admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:25.840254 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:25.887212 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4730c20-8894-4cb6-9fb0-8fdd172e190b admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:25.923320 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90a4f7-dec3-4173-95e7-701a2598209a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:25.924998 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90a4f7-dec3-4173-95e7-701a2598209a admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:25.925276 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90a4f7-dec3-4173-95e7-701a2598209a admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:25.925552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90a4f7-dec3-4173-95e7-701a2598209a admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:25.929004 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90a4f7-dec3-4173-95e7-701a2598209a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:25.931700 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90a4f7-dec3-4173-95e7-701a2598209a admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:26.000580 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1275/2542] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:58:25 2022] POST /identity/v3/projects => generated 383 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:58:26.002013 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55971377-8e76-494f-9471-c820a4d2d1af None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:26.026258 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55971377-8e76-494f-9471-c820a4d2d1af None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:26.206208 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f2b4c-8637-415c-b853-7094fe7e0b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:26.206863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f2b4c-8637-415c-b853-7094fe7e0b08 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:26.207179 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f2b4c-8637-415c-b853-7094fe7e0b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:26.207461 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f2b4c-8637-415c-b853-7094fe7e0b08 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:26.208121 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f2b4c-8637-415c-b853-7094fe7e0b08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:26.210307 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f2b4c-8637-415c-b853-7094fe7e0b08 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:26.256699 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1268/2543] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:58:26 2022] POST /identity/v3/users => generated 298 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:58:26.260873 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90a4f7-dec3-4173-95e7-701a2598209a admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:26.275336 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90a4f7-dec3-4173-95e7-701a2598209a admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:26.285892 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd46ef3-b252-453e-b954-3be1fa78ce12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:26.286439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd46ef3-b252-453e-b954-3be1fa78ce12 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:26.286718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd46ef3-b252-453e-b954-3be1fa78ce12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:26.286992 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd46ef3-b252-453e-b954-3be1fa78ce12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:26.287673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd46ef3-b252-453e-b954-3be1fa78ce12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:26.289849 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd46ef3-b252-453e-b954-3be1fa78ce12 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:26.311369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1276/2544] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:58:26.315522 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f2b4c-8637-415c-b853-7094fe7e0b08 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:26.330845 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f2b4c-8637-415c-b853-7094fe7e0b08 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:26.348006 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e137d21-4237-4fa7-8953-704a0d235d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:26.348699 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e137d21-4237-4fa7-8953-704a0d235d42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:26.348976 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e137d21-4237-4fa7-8953-704a0d235d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:26.349249 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e137d21-4237-4fa7-8953-704a0d235d42 admin admin] PATH_INFO: `/v3/projects/a98e95afb6f94cd4a99c20ec2a8a9811/users/2d0225ffb1d34740a57fab6d4c544af6/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:26.379854 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e137d21-4237-4fa7-8953-704a0d235d42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a98e95afb6f94cd4a99c20ec2a8a9811, user_id=2d0225ffb1d34740a57fab6d4c544af6, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:26.392125 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e137d21-4237-4fa7-8953-704a0d235d42 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:26.430823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1269/2545] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:58:26 2022] PUT /identity/v3/projects/a98e95afb6f94cd4a99c20ec2a8a9811/users/2d0225ffb1d34740a57fab6d4c544af6/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:26.434562 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd46ef3-b252-453e-b954-3be1fa78ce12 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:26.452032 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd46ef3-b252-453e-b954-3be1fa78ce12 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:26.599274 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4ee57-8456-4f56-bc51-46984bacea25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:26.601220 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4ee57-8456-4f56-bc51-46984bacea25 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:26.601634 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4ee57-8456-4f56-bc51-46984bacea25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:26.602043 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4ee57-8456-4f56-bc51-46984bacea25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:26.603084 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d4ee57-8456-4f56-bc51-46984bacea25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:26.608431 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d4ee57-8456-4f56-bc51-46984bacea25 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:26.636663 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1277/2546] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:58:26.638745 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e137d21-4237-4fa7-8953-704a0d235d42 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:26.656246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e137d21-4237-4fa7-8953-704a0d235d42 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:26.673258 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f1bb9-ac92-4ade-9196-52c83b112c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:26.674681 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f1bb9-ac92-4ade-9196-52c83b112c1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:26.675137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f1bb9-ac92-4ade-9196-52c83b112c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:26.676259 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f1bb9-ac92-4ade-9196-52c83b112c1e admin admin] PATH_INFO: `/v3/projects/a98e95afb6f94cd4a99c20ec2a8a9811/users/2d0225ffb1d34740a57fab6d4c544af6/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:26.679565 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f1bb9-ac92-4ade-9196-52c83b112c1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a98e95afb6f94cd4a99c20ec2a8a9811, user_id=2d0225ffb1d34740a57fab6d4c544af6, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:26.686247 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f1bb9-ac92-4ade-9196-52c83b112c1e admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:26.706116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1270/2547] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:58:26 2022] PUT /identity/v3/projects/a98e95afb6f94cd4a99c20ec2a8a9811/users/2d0225ffb1d34740a57fab6d4c544af6/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:26.724261 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18575c96-c9db-4619-b7a3-6999c54610e4 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:26.724682 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18575c96-c9db-4619-b7a3-6999c54610e4 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:26.725221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18575c96-c9db-4619-b7a3-6999c54610e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:26.780980 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-18575c96-c9db-4619-b7a3-6999c54610e4 None None] MFA Rules not processed for user `2d0225ffb1d34740a57fab6d4c544af6`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:26.783400 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-18575c96-c9db-4619-b7a3-6999c54610e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:26.982008 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1278/2548] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:26.994118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f1bb9-ac92-4ade-9196-52c83b112c1e admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:26.996461 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-223f1bb9-ac92-4ade-9196-52c83b112c1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:27.084766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f1bb9-ac92-4ade-9196-52c83b112c1e admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:27.091304 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e901d0dc-95e3-4bee-b340-18169e268896 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:27.091954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901d0dc-95e3-4bee-b340-18169e268896 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:27.092235 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901d0dc-95e3-4bee-b340-18169e268896 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:27.092509 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901d0dc-95e3-4bee-b340-18169e268896 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:27.111582 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e901d0dc-95e3-4bee-b340-18169e268896 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:27.113894 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e901d0dc-95e3-4bee-b340-18169e268896 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:27.137320 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1271/2549] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:26 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:58:27.161569 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18575c96-c9db-4619-b7a3-6999c54610e4 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:27.277559 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:27.316523 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18575c96-c9db-4619-b7a3-6999c54610e4 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:27.324435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7c10d-d7f3-4d2b-99ee-a58ffd9c420b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:27.325853 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7c10d-d7f3-4d2b-99ee-a58ffd9c420b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:27.326467 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7c10d-d7f3-4d2b-99ee-a58ffd9c420b admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:27.326895 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7c10d-d7f3-4d2b-99ee-a58ffd9c420b admin admin] PATH_INFO: `/v3/users/be4920c391b94cb39b1b54ddd7840326` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:27.328294 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc7c10d-d7f3-4d2b-99ee-a58ffd9c420b admin admin] RBAC: Authorizing `identity:delete_user(user_id=be4920c391b94cb39b1b54ddd7840326)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:27.331877 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc7c10d-d7f3-4d2b-99ee-a58ffd9c420b admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:27.428454 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-7dc7c10d-d7f3-4d2b-99ee-a58ffd9c420b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be4920c391b94cb39b1b54ddd7840326'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:27.448775 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-7dc7c10d-d7f3-4d2b-99ee-a58ffd9c420b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be4920c391b94cb39b1b54ddd7840326'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:27.474473 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-7dc7c10d-d7f3-4d2b-99ee-a58ffd9c420b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be4920c391b94cb39b1b54ddd7840326'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:27.478079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-7dc7c10d-d7f3-4d2b-99ee-a58ffd9c420b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be4920c391b94cb39b1b54ddd7840326'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:27.479528 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-7dc7c10d-d7f3-4d2b-99ee-a58ffd9c420b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be4920c391b94cb39b1b54ddd7840326'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:27.534065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1279/2550] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:58:27 2022] DELETE /identity/v3/users/be4920c391b94cb39b1b54ddd7840326 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:27.534927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d76ec-d74f-4d9d-8f58-8923349e9b6e None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:27.535251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d76ec-d74f-4d9d-8f58-8923349e9b6e None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:27.535525 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d76ec-d74f-4d9d-8f58-8923349e9b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:27.567868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-356d76ec-d74f-4d9d-8f58-8923349e9b6e None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:27.570675 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-356d76ec-d74f-4d9d-8f58-8923349e9b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:27.642450 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:27.675881 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1272/2551] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:27.722451 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7c10d-d7f3-4d2b-99ee-a58ffd9c420b admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:27.724141 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-7dc7c10d-d7f3-4d2b-99ee-a58ffd9c420b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:27.897820 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7c10d-d7f3-4d2b-99ee-a58ffd9c420b admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:27.909276 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476f033-3d65-4699-9dba-d06169dc8efc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:27.909917 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476f033-3d65-4699-9dba-d06169dc8efc service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:27.910198 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476f033-3d65-4699-9dba-d06169dc8efc service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:27.910465 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476f033-3d65-4699-9dba-d06169dc8efc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:27.935841 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476f033-3d65-4699-9dba-d06169dc8efc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:27.937348 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476f033-3d65-4699-9dba-d06169dc8efc service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:27.960182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1280/2552] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:58:28.506692 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d76ec-d74f-4d9d-8f58-8923349e9b6e None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:28.509805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-356d76ec-d74f-4d9d-8f58-8923349e9b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:28.524228 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d76ec-d74f-4d9d-8f58-8923349e9b6e None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:28.530345 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de0b76d-a093-4909-aaa4-ecdb267b6bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:28.532317 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0b76d-a093-4909-aaa4-ecdb267b6bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:28.532711 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0b76d-a093-4909-aaa4-ecdb267b6bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:28.533084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0b76d-a093-4909-aaa4-ecdb267b6bfd admin admin] PATH_INFO: `/v3/projects/5b58f1c5c2b94ca6b1ea27cb83454378` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:28.534546 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de0b76d-a093-4909-aaa4-ecdb267b6bfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b58f1c5c2b94ca6b1ea27cb83454378)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:28.536873 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de0b76d-a093-4909-aaa4-ecdb267b6bfd admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:28.563670 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a666b66-6e20-43cc-82e4-41918ddddcd3 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:28.563979 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a666b66-6e20-43cc-82e4-41918ddddcd3 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:28.564256 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a666b66-6e20-43cc-82e4-41918ddddcd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:28.623413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-4de0b76d-a093-4909-aaa4-ecdb267b6bfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b58f1c5c2b94ca6b1ea27cb83454378', 'request_id': 'req-4de0b76d-a093-4909-aaa4-ecdb267b6bfd'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:28.645151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-4de0b76d-a093-4909-aaa4-ecdb267b6bfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b58f1c5c2b94ca6b1ea27cb83454378', 'request_id': 'req-4de0b76d-a093-4909-aaa4-ecdb267b6bfd'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:28.665892 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-4de0b76d-a093-4909-aaa4-ecdb267b6bfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b58f1c5c2b94ca6b1ea27cb83454378', 'request_id': 'req-4de0b76d-a093-4909-aaa4-ecdb267b6bfd'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:28.668711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-3a666b66-6e20-43cc-82e4-41918ddddcd3 None None] MFA Rules not processed for user `5f99a7b91cd54ba7a9bd79484a7fb436`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:28.670157 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-3a666b66-6e20-43cc-82e4-41918ddddcd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:28.694011 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-4de0b76d-a093-4909-aaa4-ecdb267b6bfd admin admin] The token cache is being invalidate because project 5b58f1c5c2b94ca6b1ea27cb83454378 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:58:28.695677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-4de0b76d-a093-4909-aaa4-ecdb267b6bfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:28.697067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-4de0b76d-a093-4909-aaa4-ecdb267b6bfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:28.702198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1273/2553] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:58:28 2022] DELETE /identity/v3/projects/5b58f1c5c2b94ca6b1ea27cb83454378 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:28.707563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de0b76d-a093-4909-aaa4-ecdb267b6bfd admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:28.709285 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-4de0b76d-a093-4909-aaa4-ecdb267b6bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:28.759305 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:28.817273 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1281/2554] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:28 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:28.827324 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a666b66-6e20-43cc-82e4-41918ddddcd3 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:28.829129 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-3a666b66-6e20-43cc-82e4-41918ddddcd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:28.854427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de0b76d-a093-4909-aaa4-ecdb267b6bfd admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:28.860287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d9d423-400b-4ec5-bdf8-c10e789c21d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:28.861207 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9d423-400b-4ec5-bdf8-c10e789c21d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:28.861528 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9d423-400b-4ec5-bdf8-c10e789c21d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:28.861939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9d423-400b-4ec5-bdf8-c10e789c21d5 admin admin] PATH_INFO: `/v3/users/2d0225ffb1d34740a57fab6d4c544af6` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:28.863461 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d9d423-400b-4ec5-bdf8-c10e789c21d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d0225ffb1d34740a57fab6d4c544af6)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:28.865520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d9d423-400b-4ec5-bdf8-c10e789c21d5 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:28.951197 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-48d9d423-400b-4ec5-bdf8-c10e789c21d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d0225ffb1d34740a57fab6d4c544af6'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:28.964627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a666b66-6e20-43cc-82e4-41918ddddcd3 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:28.974425 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d6944-9526-4190-bcf0-c96270ffc254 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:28.974881 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-48d9d423-400b-4ec5-bdf8-c10e789c21d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d0225ffb1d34740a57fab6d4c544af6'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:28.975995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d6944-9526-4190-bcf0-c96270ffc254 service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:28.977432 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d6944-9526-4190-bcf0-c96270ffc254 service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:28.977709 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d6944-9526-4190-bcf0-c96270ffc254 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:28.980100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-1a4d6944-9526-4190-bcf0-c96270ffc254 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:28.983806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:28.995503 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-48d9d423-400b-4ec5-bdf8-c10e789c21d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d0225ffb1d34740a57fab6d4c544af6'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:29.007058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-48d9d423-400b-4ec5-bdf8-c10e789c21d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d0225ffb1d34740a57fab6d4c544af6'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:29.007058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-48d9d423-400b-4ec5-bdf8-c10e789c21d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d0225ffb1d34740a57fab6d4c544af6'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:29.020374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1274/2555] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:58:28 2022] DELETE /identity/v3/users/2d0225ffb1d34740a57fab6d4c544af6 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:58:29.083853 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d6944-9526-4190-bcf0-c96270ffc254 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:29.087294 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d6944-9526-4190-bcf0-c96270ffc254 service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:29.107147 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1282/2556] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:58:29.418449 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d9d423-400b-4ec5-bdf8-c10e789c21d5 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:29.420257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-48d9d423-400b-4ec5-bdf8-c10e789c21d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:29.535835 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d9d423-400b-4ec5-bdf8-c10e789c21d5 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:29.548679 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4614f1f-0819-437d-8b2e-975d2b44bcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:29.550288 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4614f1f-0819-437d-8b2e-975d2b44bcb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:29.550670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4614f1f-0819-437d-8b2e-975d2b44bcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:29.551293 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4614f1f-0819-437d-8b2e-975d2b44bcb0 admin admin] PATH_INFO: `/v3/projects/a98e95afb6f94cd4a99c20ec2a8a9811` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:29.552592 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4614f1f-0819-437d-8b2e-975d2b44bcb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a98e95afb6f94cd4a99c20ec2a8a9811)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:29.556163 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4614f1f-0819-437d-8b2e-975d2b44bcb0 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:29.644252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b4614f1f-0819-437d-8b2e-975d2b44bcb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a98e95afb6f94cd4a99c20ec2a8a9811', 'request_id': 'req-b4614f1f-0819-437d-8b2e-975d2b44bcb0'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:29.655198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b4614f1f-0819-437d-8b2e-975d2b44bcb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a98e95afb6f94cd4a99c20ec2a8a9811', 'request_id': 'req-b4614f1f-0819-437d-8b2e-975d2b44bcb0'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:29.664767 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b4614f1f-0819-437d-8b2e-975d2b44bcb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a98e95afb6f94cd4a99c20ec2a8a9811', 'request_id': 'req-b4614f1f-0819-437d-8b2e-975d2b44bcb0'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:29.686309 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b4614f1f-0819-437d-8b2e-975d2b44bcb0 admin admin] The token cache is being invalidate because project a98e95afb6f94cd4a99c20ec2a8a9811 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:58:29.687646 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b4614f1f-0819-437d-8b2e-975d2b44bcb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:29.688627 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b4614f1f-0819-437d-8b2e-975d2b44bcb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:29.690393 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1275/2557] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:58:29 2022] DELETE /identity/v3/projects/a98e95afb6f94cd4a99c20ec2a8a9811 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:58:29.696624 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6d9ad-01d0-4412-a6ed-211197c68ccb None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:29.700788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6d9ad-01d0-4412-a6ed-211197c68ccb None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:29.701834 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6d9ad-01d0-4412-a6ed-211197c68ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:29.745211 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-97c6d9ad-01d0-4412-a6ed-211197c68ccb None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:29.747207 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-97c6d9ad-01d0-4412-a6ed-211197c68ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:29.933301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1283/2558] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:29.941037 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8267b38d-f64f-4e6e-886c-69d459bad64f None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:29.941267 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8267b38d-f64f-4e6e-886c-69d459bad64f None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:29.941577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8267b38d-f64f-4e6e-886c-69d459bad64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:30.016914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-8267b38d-f64f-4e6e-886c-69d459bad64f None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:30.019114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-8267b38d-f64f-4e6e-886c-69d459bad64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:30.043229 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1276/2559] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:30.047518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6d9ad-01d0-4412-a6ed-211197c68ccb None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:30.061690 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6d9ad-01d0-4412-a6ed-211197c68ccb None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:30.071931 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd8bbb-1c6d-4fd8-be0d-5a12dc748aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:30.072641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd8bbb-1c6d-4fd8-be0d-5a12dc748aaf admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:30.072927 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd8bbb-1c6d-4fd8-be0d-5a12dc748aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:30.073338 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd8bbb-1c6d-4fd8-be0d-5a12dc748aaf admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:30.074009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd8bbb-1c6d-4fd8-be0d-5a12dc748aaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:58:30.074318 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd8bbb-1c6d-4fd8-be0d-5a12dc748aaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:30.085091 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd8bbb-1c6d-4fd8-be0d-5a12dc748aaf admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:30.105891 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1284/2560] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:58:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:58:30.116132 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8267b38d-f64f-4e6e-886c-69d459bad64f None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:30.128512 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8267b38d-f64f-4e6e-886c-69d459bad64f None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:30.134750 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9affda-8ba9-4549-8fd5-962567d0886a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:30.135678 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9affda-8ba9-4549-8fd5-962567d0886a admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:30.136316 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9affda-8ba9-4549-8fd5-962567d0886a admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:30.136767 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9affda-8ba9-4549-8fd5-962567d0886a admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:30.137565 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9affda-8ba9-4549-8fd5-962567d0886a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:30.139758 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9affda-8ba9-4549-8fd5-962567d0886a admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:30.206665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1277/2561] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:58:30 2022] POST /identity/v3/projects => generated 393 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:58:30.207908 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd8bbb-1c6d-4fd8-be0d-5a12dc748aaf admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:30.231528 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd8bbb-1c6d-4fd8-be0d-5a12dc748aaf admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:30.246017 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:30.357132 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0257f8c-093a-4138-abad-f7092d42f738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:30.358442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0257f8c-093a-4138-abad-f7092d42f738 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:30.359170 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0257f8c-093a-4138-abad-f7092d42f738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:30.359755 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0257f8c-093a-4138-abad-f7092d42f738 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:30.360158 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0257f8c-093a-4138-abad-f7092d42f738 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:30.362849 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0257f8c-093a-4138-abad-f7092d42f738 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:30.414553 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1285/2562] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:58:30 2022] POST /identity/v3/users => generated 297 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:58:30.414995 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9affda-8ba9-4549-8fd5-962567d0886a admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:30.428790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9affda-8ba9-4549-8fd5-962567d0886a admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:30.436386 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a85832-d0b8-4402-803c-068ec91b5cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:30.437233 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a85832-d0b8-4402-803c-068ec91b5cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:30.437602 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a85832-d0b8-4402-803c-068ec91b5cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:30.438682 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a85832-d0b8-4402-803c-068ec91b5cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:30.443346 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a85832-d0b8-4402-803c-068ec91b5cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:30.448560 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a85832-d0b8-4402-803c-068ec91b5cc7 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:30.492430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1278/2563] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:58:30.492958 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0257f8c-093a-4138-abad-f7092d42f738 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:30.504261 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0257f8c-093a-4138-abad-f7092d42f738 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:30.515207 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b33c7-49f6-46b7-8122-0fde186a2275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:30.516004 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b33c7-49f6-46b7-8122-0fde186a2275 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:30.516297 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b33c7-49f6-46b7-8122-0fde186a2275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:30.516628 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b33c7-49f6-46b7-8122-0fde186a2275 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:30.517354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b33c7-49f6-46b7-8122-0fde186a2275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:30.519772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b33c7-49f6-46b7-8122-0fde186a2275 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:30.532581 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a85832-d0b8-4402-803c-068ec91b5cc7 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:30.535161 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-24a85832-d0b8-4402-803c-068ec91b5cc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:30.568685 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1286/2564] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:58:30.570789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b33c7-49f6-46b7-8122-0fde186a2275 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:30.602207 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b33c7-49f6-46b7-8122-0fde186a2275 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:30.624453 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e071deb-326a-46bb-917e-c5a124b33b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:30.625936 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e071deb-326a-46bb-917e-c5a124b33b29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:30.626418 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e071deb-326a-46bb-917e-c5a124b33b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:30.626701 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e071deb-326a-46bb-917e-c5a124b33b29 admin admin] PATH_INFO: `/v3/projects/17d0b6c73b784e2f91496e5bed34b43f/users/568e6e9b4e114552a988789d2ba10056/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:30.646996 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e071deb-326a-46bb-917e-c5a124b33b29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17d0b6c73b784e2f91496e5bed34b43f, user_id=568e6e9b4e114552a988789d2ba10056, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:30.652022 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e071deb-326a-46bb-917e-c5a124b33b29 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:30.667412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:30.686319 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1287/2565] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:58:30 2022] PUT /identity/v3/projects/17d0b6c73b784e2f91496e5bed34b43f/users/568e6e9b4e114552a988789d2ba10056/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:30.699071 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202a4dd-95ca-49f4-9a1c-80246e4e354e None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:30.699377 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202a4dd-95ca-49f4-9a1c-80246e4e354e None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:30.701442 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202a4dd-95ca-49f4-9a1c-80246e4e354e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:30.759734 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a85832-d0b8-4402-803c-068ec91b5cc7 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:30.767528 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-3202a4dd-95ca-49f4-9a1c-80246e4e354e None None] MFA Rules not processed for user `568e6e9b4e114552a988789d2ba10056`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:30.770312 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-3202a4dd-95ca-49f4-9a1c-80246e4e354e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:30.889490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d23ce-cf3d-4574-b050-92dcfa1c30cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:30.890172 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d23ce-cf3d-4574-b050-92dcfa1c30cf service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:30.890457 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d23ce-cf3d-4574-b050-92dcfa1c30cf service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:30.890810 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d23ce-cf3d-4574-b050-92dcfa1c30cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:30.893159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-ea1d23ce-cf3d-4574-b050-92dcfa1c30cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:30.922961 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d23ce-cf3d-4574-b050-92dcfa1c30cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:30.927323 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d23ce-cf3d-4574-b050-92dcfa1c30cf service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:30.928704 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1288/2566] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:30 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:30.933946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202a4dd-95ca-49f4-9a1c-80246e4e354e None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:30.963133 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1279/2567] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:58:31.072382 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:31.073247 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198e451-e6ee-4d16-895f-0bb41dd706f0 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:31.074014 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198e451-e6ee-4d16-895f-0bb41dd706f0 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:31.074178 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198e451-e6ee-4d16-895f-0bb41dd706f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:31.081244 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202a4dd-95ca-49f4-9a1c-80246e4e354e None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:31.087975 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63c64a-3f8a-4c8e-aeb8-1d6663ddba94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:31.088613 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63c64a-3f8a-4c8e-aeb8-1d6663ddba94 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:31.088904 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63c64a-3f8a-4c8e-aeb8-1d6663ddba94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:31.089232 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63c64a-3f8a-4c8e-aeb8-1d6663ddba94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:31.090315 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63c64a-3f8a-4c8e-aeb8-1d6663ddba94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:31.093374 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63c64a-3f8a-4c8e-aeb8-1d6663ddba94 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:31.099103 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-7198e451-e6ee-4d16-895f-0bb41dd706f0 None None] MFA Rules not processed for user `d69d5ca56a8e43ca9d91920684f2dac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:31.100422 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-7198e451-e6ee-4d16-895f-0bb41dd706f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:31.142518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1289/2568] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:58:30 2022] POST /identity/v3/projects => generated 393 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:58:31.156084 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63c64a-3f8a-4c8e-aeb8-1d6663ddba94 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:31.180368 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1280/2569] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:31.242664 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63c64a-3f8a-4c8e-aeb8-1d6663ddba94 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:31.248087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae543b0-82f7-4a94-b89a-fc442f323a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:31.248704 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae543b0-82f7-4a94-b89a-fc442f323a60 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:31.248989 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae543b0-82f7-4a94-b89a-fc442f323a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:31.249261 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae543b0-82f7-4a94-b89a-fc442f323a60 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:31.249896 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae543b0-82f7-4a94-b89a-fc442f323a60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:31.253486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae543b0-82f7-4a94-b89a-fc442f323a60 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:31.280516 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1290/2570] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:58:31 2022] POST /identity/v3/users => generated 303 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:58:31.284743 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198e451-e6ee-4d16-895f-0bb41dd706f0 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:31.297130 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198e451-e6ee-4d16-895f-0bb41dd706f0 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:31.302762 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491a005-5646-4fcd-bf44-39f07145b3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:31.303408 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491a005-5646-4fcd-bf44-39f07145b3f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:31.303653 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491a005-5646-4fcd-bf44-39f07145b3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:31.303928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491a005-5646-4fcd-bf44-39f07145b3f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:31.305279 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2491a005-5646-4fcd-bf44-39f07145b3f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:31.306999 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2491a005-5646-4fcd-bf44-39f07145b3f3 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:31.322024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1281/2571] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:58:31.323638 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae543b0-82f7-4a94-b89a-fc442f323a60 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:31.335137 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae543b0-82f7-4a94-b89a-fc442f323a60 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:31.344050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad211a1-2b0c-4ad1-93fe-e95e3cb0560a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:31.345063 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad211a1-2b0c-4ad1-93fe-e95e3cb0560a admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:31.345463 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad211a1-2b0c-4ad1-93fe-e95e3cb0560a admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:31.345981 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad211a1-2b0c-4ad1-93fe-e95e3cb0560a admin admin] PATH_INFO: `/v3/projects/3a432b568f7848b991e5b491d30cbe62/users/4a76e4b163e047369a5b6f4bb1c9348b/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:31.365128 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad211a1-2b0c-4ad1-93fe-e95e3cb0560a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a432b568f7848b991e5b491d30cbe62, user_id=4a76e4b163e047369a5b6f4bb1c9348b, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:31.368811 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad211a1-2b0c-4ad1-93fe-e95e3cb0560a admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:31.387267 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1291/2572] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:58:31 2022] PUT /identity/v3/projects/3a432b568f7848b991e5b491d30cbe62/users/4a76e4b163e047369a5b6f4bb1c9348b/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:58:31.391470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491a005-5646-4fcd-bf44-39f07145b3f3 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:31.406899 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491a005-5646-4fcd-bf44-39f07145b3f3 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:31.475998 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:31.487086 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c19477-3269-49df-af8a-59b2c5017121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:31.488474 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c19477-3269-49df-af8a-59b2c5017121 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:31.488764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c19477-3269-49df-af8a-59b2c5017121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:31.489926 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c19477-3269-49df-af8a-59b2c5017121 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:31.490717 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c19477-3269-49df-af8a-59b2c5017121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:31.492420 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c19477-3269-49df-af8a-59b2c5017121 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:31.510705 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1282/2573] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:58:31.516046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad211a1-2b0c-4ad1-93fe-e95e3cb0560a admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:31.528148 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad211a1-2b0c-4ad1-93fe-e95e3cb0560a admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:31.533672 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512042c-ede7-4c02-93b7-486655c36bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:31.534564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512042c-ede7-4c02-93b7-486655c36bec admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:31.534942 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512042c-ede7-4c02-93b7-486655c36bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:31.535355 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512042c-ede7-4c02-93b7-486655c36bec admin admin] PATH_INFO: `/v3/projects/3a432b568f7848b991e5b491d30cbe62/users/4a76e4b163e047369a5b6f4bb1c9348b/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:31.537868 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512042c-ede7-4c02-93b7-486655c36bec admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a432b568f7848b991e5b491d30cbe62, user_id=4a76e4b163e047369a5b6f4bb1c9348b, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:31.541877 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512042c-ede7-4c02-93b7-486655c36bec admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:31.554938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1292/2574] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:58:31 2022] PUT /identity/v3/projects/3a432b568f7848b991e5b491d30cbe62/users/4a76e4b163e047369a5b6f4bb1c9348b/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:31.565745 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea455ec-c106-44e2-b76a-6ecfc037534d None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:31.566033 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea455ec-c106-44e2-b76a-6ecfc037534d None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:31.566333 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea455ec-c106-44e2-b76a-6ecfc037534d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:31.602155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-eea455ec-c106-44e2-b76a-6ecfc037534d None None] MFA Rules not processed for user `4a76e4b163e047369a5b6f4bb1c9348b`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:31.603873 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-eea455ec-c106-44e2-b76a-6ecfc037534d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:31.738951 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1283/2575] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:31 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:31.756194 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512042c-ede7-4c02-93b7-486655c36bec admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:31.759895 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b512042c-ede7-4c02-93b7-486655c36bec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:31.841792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512042c-ede7-4c02-93b7-486655c36bec admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:31.845846 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe67929-f8ef-46b5-9b27-4feb52b9e04b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:31.846538 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe67929-f8ef-46b5-9b27-4feb52b9e04b service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:31.846899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe67929-f8ef-46b5-9b27-4feb52b9e04b service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:31.847263 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe67929-f8ef-46b5-9b27-4feb52b9e04b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:31.861005 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe67929-f8ef-46b5-9b27-4feb52b9e04b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:31.862720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe67929-f8ef-46b5-9b27-4feb52b9e04b service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:31.875434 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1293/2576] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:31 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:58:31.927801 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea455ec-c106-44e2-b76a-6ecfc037534d None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:31.930315 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-eea455ec-c106-44e2-b76a-6ecfc037534d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:31.946230 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea455ec-c106-44e2-b76a-6ecfc037534d None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:31.953812 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f7905-028a-4440-a3a0-6ade99ac928c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:31.954515 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f7905-028a-4440-a3a0-6ade99ac928c service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:31.954866 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f7905-028a-4440-a3a0-6ade99ac928c service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:31.955939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f7905-028a-4440-a3a0-6ade99ac928c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:32.070575 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f7905-028a-4440-a3a0-6ade99ac928c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:32.072122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f7905-028a-4440-a3a0-6ade99ac928c service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:32.083341 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1284/2577] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:31 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:58:32.088473 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1590c80-0dd0-4219-bfd6-a2ce10aaadf6 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:32.089005 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1590c80-0dd0-4219-bfd6-a2ce10aaadf6 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:32.089005 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1590c80-0dd0-4219-bfd6-a2ce10aaadf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:32.156462 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:32.167139 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-a1590c80-0dd0-4219-bfd6-a2ce10aaadf6 None None] MFA Rules not processed for user `a8fb7a9acefd4b418c992661037c2ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:32.168917 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a1590c80-0dd0-4219-bfd6-a2ce10aaadf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:32.282427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1294/2578] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 31 00:58:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:32.407986 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f7905-028a-4440-a3a0-6ade99ac928c service cinder] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:32.409691 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-7e8f7905-028a-4440-a3a0-6ade99ac928c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:32.603256 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:32.627385 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f7905-028a-4440-a3a0-6ade99ac928c service cinder] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:32.639320 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756211ab-c49e-446b-a434-55d7f230a31a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd3cfbc8ebd4c5a83bef09efe10bc91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:32.640600 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756211ab-c49e-446b-a434-55d7f230a31a service swift] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:32.641069 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756211ab-c49e-446b-a434-55d7f230a31a service swift] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:32.641570 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756211ab-c49e-446b-a434-55d7f230a31a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:32.658448 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756211ab-c49e-446b-a434-55d7f230a31a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:32.660874 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756211ab-c49e-446b-a434-55d7f230a31a service swift] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:32.677105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1285/2579] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 31 00:58:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:58:35.017113 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1590c80-0dd0-4219-bfd6-a2ce10aaadf6 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:35.018627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a1590c80-0dd0-4219-bfd6-a2ce10aaadf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:35.110244 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1590c80-0dd0-4219-bfd6-a2ce10aaadf6 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:35.131114 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19038d-825c-4850-83c6-8df297fac070 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:35.131114 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19038d-825c-4850-83c6-8df297fac070 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:35.131114 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19038d-825c-4850-83c6-8df297fac070 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:35.131114 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19038d-825c-4850-83c6-8df297fac070 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:35.131885 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-3b19038d-825c-4850-83c6-8df297fac070 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:35.259655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19038d-825c-4850-83c6-8df297fac070 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:35.261780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19038d-825c-4850-83c6-8df297fac070 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:35.276007 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1295/2580] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:58:36.489887 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756211ab-c49e-446b-a434-55d7f230a31a service swift] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:36.512468 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756211ab-c49e-446b-a434-55d7f230a31a service swift] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:36.521230 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d8689-db21-483e-a1fd-ecd63dccc5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:36.521890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d8689-db21-483e-a1fd-ecd63dccc5b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:36.522480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d8689-db21-483e-a1fd-ecd63dccc5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:36.522793 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d8689-db21-483e-a1fd-ecd63dccc5b6 admin admin] PATH_INFO: `/v3/users/568e6e9b4e114552a988789d2ba10056` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:36.524741 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d8689-db21-483e-a1fd-ecd63dccc5b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=568e6e9b4e114552a988789d2ba10056)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:36.526684 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d8689-db21-483e-a1fd-ecd63dccc5b6 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:36.629094 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f79d8689-db21-483e-a1fd-ecd63dccc5b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '568e6e9b4e114552a988789d2ba10056'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:36.664944 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f79d8689-db21-483e-a1fd-ecd63dccc5b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '568e6e9b4e114552a988789d2ba10056'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:36.673659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f79d8689-db21-483e-a1fd-ecd63dccc5b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '568e6e9b4e114552a988789d2ba10056'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:36.715978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f79d8689-db21-483e-a1fd-ecd63dccc5b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '568e6e9b4e114552a988789d2ba10056'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:36.716857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-f79d8689-db21-483e-a1fd-ecd63dccc5b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '568e6e9b4e114552a988789d2ba10056'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:36.745774 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1286/2581] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:58:36 2022] DELETE /identity/v3/users/568e6e9b4e114552a988789d2ba10056 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:36.750426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c188f7e-c551-4eaf-b87c-c0ce7e063090 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:36.750967 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c188f7e-c551-4eaf-b87c-c0ce7e063090 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:36.751092 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c188f7e-c551-4eaf-b87c-c0ce7e063090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:36.783945 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-5c188f7e-c551-4eaf-b87c-c0ce7e063090 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:36.787393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-5c188f7e-c551-4eaf-b87c-c0ce7e063090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:36.894975 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:36.920014 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1296/2582] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:36.929954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d8689-db21-483e-a1fd-ecd63dccc5b6 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:36.935441 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f79d8689-db21-483e-a1fd-ecd63dccc5b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:37.041114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d8689-db21-483e-a1fd-ecd63dccc5b6 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:37.045297 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57525d20-856e-451e-9131-bbe619c8ba68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:37.046211 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57525d20-856e-451e-9131-bbe619c8ba68 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:37.047334 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57525d20-856e-451e-9131-bbe619c8ba68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:37.047748 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57525d20-856e-451e-9131-bbe619c8ba68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:37.071490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57525d20-856e-451e-9131-bbe619c8ba68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:37.072971 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57525d20-856e-451e-9131-bbe619c8ba68 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:37.111263 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1287/2583] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:58:37.717464 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c188f7e-c551-4eaf-b87c-c0ce7e063090 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:37.718927 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-5c188f7e-c551-4eaf-b87c-c0ce7e063090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:37.733271 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c188f7e-c551-4eaf-b87c-c0ce7e063090 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:37.742794 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c49c8f-e8e5-4550-a42f-d773cf9c386d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:37.743762 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c49c8f-e8e5-4550-a42f-d773cf9c386d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:37.744247 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c49c8f-e8e5-4550-a42f-d773cf9c386d admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:37.744685 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c49c8f-e8e5-4550-a42f-d773cf9c386d admin admin] PATH_INFO: `/v3/projects/17d0b6c73b784e2f91496e5bed34b43f` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:37.746032 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c49c8f-e8e5-4550-a42f-d773cf9c386d admin admin] RBAC: Authorizing `identity:delete_project(project_id=17d0b6c73b784e2f91496e5bed34b43f)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:37.748211 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c49c8f-e8e5-4550-a42f-d773cf9c386d admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:37.834042 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-e1c49c8f-e8e5-4550-a42f-d773cf9c386d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17d0b6c73b784e2f91496e5bed34b43f', 'request_id': 'req-e1c49c8f-e8e5-4550-a42f-d773cf9c386d'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:37.846863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-e1c49c8f-e8e5-4550-a42f-d773cf9c386d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17d0b6c73b784e2f91496e5bed34b43f', 'request_id': 'req-e1c49c8f-e8e5-4550-a42f-d773cf9c386d'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:37.855655 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-e1c49c8f-e8e5-4550-a42f-d773cf9c386d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17d0b6c73b784e2f91496e5bed34b43f', 'request_id': 'req-e1c49c8f-e8e5-4550-a42f-d773cf9c386d'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:37.890612 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-e1c49c8f-e8e5-4550-a42f-d773cf9c386d admin admin] The token cache is being invalidate because project 17d0b6c73b784e2f91496e5bed34b43f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:58:37.891682 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-e1c49c8f-e8e5-4550-a42f-d773cf9c386d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:37.893198 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-e1c49c8f-e8e5-4550-a42f-d773cf9c386d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:37.895565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1297/2584] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:58:37 2022] DELETE /identity/v3/projects/17d0b6c73b784e2f91496e5bed34b43f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:37.899380 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57525d20-856e-451e-9131-bbe619c8ba68 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:37.924449 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57525d20-856e-451e-9131-bbe619c8ba68 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:38.047518 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912efdf3-3785-4e0b-93de-c4630cc09c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:38.048161 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912efdf3-3785-4e0b-93de-c4630cc09c09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:38.048443 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912efdf3-3785-4e0b-93de-c4630cc09c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:38.048720 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912efdf3-3785-4e0b-93de-c4630cc09c09 admin admin] PATH_INFO: `/v3/users/4a76e4b163e047369a5b6f4bb1c9348b` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:38.049876 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912efdf3-3785-4e0b-93de-c4630cc09c09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a76e4b163e047369a5b6f4bb1c9348b)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:38.051847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912efdf3-3785-4e0b-93de-c4630cc09c09 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:38.054861 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:38.130890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-912efdf3-3785-4e0b-93de-c4630cc09c09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a76e4b163e047369a5b6f4bb1c9348b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:38.160005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-912efdf3-3785-4e0b-93de-c4630cc09c09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a76e4b163e047369a5b6f4bb1c9348b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:38.181972 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-912efdf3-3785-4e0b-93de-c4630cc09c09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a76e4b163e047369a5b6f4bb1c9348b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:38.208609 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-912efdf3-3785-4e0b-93de-c4630cc09c09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a76e4b163e047369a5b6f4bb1c9348b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:38.208609 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-912efdf3-3785-4e0b-93de-c4630cc09c09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a76e4b163e047369a5b6f4bb1c9348b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:38.216120 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1288/2585] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:58:37 2022] DELETE /identity/v3/users/4a76e4b163e047369a5b6f4bb1c9348b => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:38.670250 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c49c8f-e8e5-4550-a42f-d773cf9c386d admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:38.671748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e1c49c8f-e8e5-4550-a42f-d773cf9c386d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:38.702464 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c49c8f-e8e5-4550-a42f-d773cf9c386d admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:38.815614 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df95f4a-963f-4bbb-8a01-0959108a022d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:38.816260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df95f4a-963f-4bbb-8a01-0959108a022d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:38.816541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df95f4a-963f-4bbb-8a01-0959108a022d admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:38.816813 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df95f4a-963f-4bbb-8a01-0959108a022d admin admin] PATH_INFO: `/v3/projects/3a432b568f7848b991e5b491d30cbe62` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:38.817959 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df95f4a-963f-4bbb-8a01-0959108a022d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a432b568f7848b991e5b491d30cbe62)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:38.820575 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df95f4a-963f-4bbb-8a01-0959108a022d admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:38.904264 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-6df95f4a-963f-4bbb-8a01-0959108a022d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a432b568f7848b991e5b491d30cbe62', 'request_id': 'req-6df95f4a-963f-4bbb-8a01-0959108a022d'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:38.910894 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-6df95f4a-963f-4bbb-8a01-0959108a022d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a432b568f7848b991e5b491d30cbe62', 'request_id': 'req-6df95f4a-963f-4bbb-8a01-0959108a022d'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:38.919130 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-6df95f4a-963f-4bbb-8a01-0959108a022d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a432b568f7848b991e5b491d30cbe62', 'request_id': 'req-6df95f4a-963f-4bbb-8a01-0959108a022d'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:38.952058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-6df95f4a-963f-4bbb-8a01-0959108a022d admin admin] The token cache is being invalidate because project 3a432b568f7848b991e5b491d30cbe62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:58:38.952419 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-6df95f4a-963f-4bbb-8a01-0959108a022d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:38.955929 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-6df95f4a-963f-4bbb-8a01-0959108a022d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:38.957518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1298/2586] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:58:38 2022] DELETE /identity/v3/projects/3a432b568f7848b991e5b491d30cbe62 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:38.962735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5731f9-c870-4495-b1ce-ad8cb34c7d7a None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:38.963275 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5731f9-c870-4495-b1ce-ad8cb34c7d7a None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:38.963381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5731f9-c870-4495-b1ce-ad8cb34c7d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:39.004451 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-ae5731f9-c870-4495-b1ce-ad8cb34c7d7a None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:39.005946 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-ae5731f9-c870-4495-b1ce-ad8cb34c7d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:39.083805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1289/2587] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:39.085674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd5228-e093-4058-bf29-f1351b03d4a5 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:39.086157 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd5228-e093-4058-bf29-f1351b03d4a5 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:39.086541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd5228-e093-4058-bf29-f1351b03d4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:39.104221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:39.120432 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-98fd5228-e093-4058-bf29-f1351b03d4a5 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:39.121861 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-98fd5228-e093-4058-bf29-f1351b03d4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:39.134410 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1299/2588] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:39.138717 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5731f9-c870-4495-b1ce-ad8cb34c7d7a None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:39.155249 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5731f9-c870-4495-b1ce-ad8cb34c7d7a None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:39.170823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a66002-e5cc-415f-9a83-e4b55dc224b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:39.171486 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a66002-e5cc-415f-9a83-e4b55dc224b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:39.171911 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a66002-e5cc-415f-9a83-e4b55dc224b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:39.172244 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a66002-e5cc-415f-9a83-e4b55dc224b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:39.172918 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a66002-e5cc-415f-9a83-e4b55dc224b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:58:39.173224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a66002-e5cc-415f-9a83-e4b55dc224b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:39.175060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a66002-e5cc-415f-9a83-e4b55dc224b6 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:39.209658 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1290/2589] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:58:39.215431 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd5228-e093-4058-bf29-f1351b03d4a5 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:39.231775 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd5228-e093-4058-bf29-f1351b03d4a5 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:39.249601 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ffd09-3dcc-4515-831a-fcde59b3de8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:39.250474 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ffd09-3dcc-4515-831a-fcde59b3de8c admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:39.250757 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ffd09-3dcc-4515-831a-fcde59b3de8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:39.251047 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ffd09-3dcc-4515-831a-fcde59b3de8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:39.251984 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ffd09-3dcc-4515-831a-fcde59b3de8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:39.254510 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ffd09-3dcc-4515-831a-fcde59b3de8c admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:39.306777 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1300/2590] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:58:39 2022] POST /identity/v3/projects => generated 399 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:58:39.311009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a66002-e5cc-415f-9a83-e4b55dc224b6 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:39.332783 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a66002-e5cc-415f-9a83-e4b55dc224b6 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:39.463991 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27dc118-bf81-4a63-89ed-4f106537538b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:39.465345 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27dc118-bf81-4a63-89ed-4f106537538b admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:39.467719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27dc118-bf81-4a63-89ed-4f106537538b admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:39.468287 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27dc118-bf81-4a63-89ed-4f106537538b admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:39.470352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27dc118-bf81-4a63-89ed-4f106537538b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:39.478930 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27dc118-bf81-4a63-89ed-4f106537538b admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:39.502808 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:39.525433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1291/2591] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:58:39 2022] POST /identity/v3/users => generated 300 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:58:39.528116 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ffd09-3dcc-4515-831a-fcde59b3de8c admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:39.542908 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ffd09-3dcc-4515-831a-fcde59b3de8c admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:39.548730 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73e794-ad8b-405b-a518-a70c6d04a61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:39.549567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73e794-ad8b-405b-a518-a70c6d04a61e admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:39.550155 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73e794-ad8b-405b-a518-a70c6d04a61e admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:39.550563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73e794-ad8b-405b-a518-a70c6d04a61e admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:39.551486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c73e794-ad8b-405b-a518-a70c6d04a61e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:39.553288 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c73e794-ad8b-405b-a518-a70c6d04a61e admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:39.571678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1301/2592] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:58:39.573131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27dc118-bf81-4a63-89ed-4f106537538b admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:39.594646 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27dc118-bf81-4a63-89ed-4f106537538b admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:39.606949 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc66536-7979-44d4-b5cc-18357f7c1256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:39.606949 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc66536-7979-44d4-b5cc-18357f7c1256 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:39.606949 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc66536-7979-44d4-b5cc-18357f7c1256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:39.606949 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc66536-7979-44d4-b5cc-18357f7c1256 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:39.607834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc66536-7979-44d4-b5cc-18357f7c1256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:39.607834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc66536-7979-44d4-b5cc-18357f7c1256 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:39.625740 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1292/2593] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:39 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:58:39.627045 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73e794-ad8b-405b-a518-a70c6d04a61e admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:39.646716 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73e794-ad8b-405b-a518-a70c6d04a61e admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:39.655704 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f2ac1-af22-4d6f-8b5e-37273faf1f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:39.657022 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f2ac1-af22-4d6f-8b5e-37273faf1f01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:39.658426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f2ac1-af22-4d6f-8b5e-37273faf1f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:39.658920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f2ac1-af22-4d6f-8b5e-37273faf1f01 admin admin] PATH_INFO: `/v3/projects/e3e7327226f34401b53130eb712049fc/users/0b7e62a06c3b46ce9b619c277c5629e1/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:39.680051 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f2ac1-af22-4d6f-8b5e-37273faf1f01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3e7327226f34401b53130eb712049fc, user_id=0b7e62a06c3b46ce9b619c277c5629e1, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:39.684097 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f2ac1-af22-4d6f-8b5e-37273faf1f01 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:39.707868 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1302/2594] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:58:39 2022] PUT /identity/v3/projects/e3e7327226f34401b53130eb712049fc/users/0b7e62a06c3b46ce9b619c277c5629e1/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:39.721430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2fd9b-c89f-4741-87ca-7e32dead79de None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:39.722231 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2fd9b-c89f-4741-87ca-7e32dead79de None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:39.722231 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2fd9b-c89f-4741-87ca-7e32dead79de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:39.771580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-f0a2fd9b-c89f-4741-87ca-7e32dead79de None None] MFA Rules not processed for user `0b7e62a06c3b46ce9b619c277c5629e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:39.773712 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f0a2fd9b-c89f-4741-87ca-7e32dead79de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:39.955206 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1293/2595] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:39 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:39.955429 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f2ac1-af22-4d6f-8b5e-37273faf1f01 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:40.104450 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f2ac1-af22-4d6f-8b5e-37273faf1f01 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:40.112871 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b6e7e-9803-4463-8fb9-aea24cbd3e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:40.113808 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b6e7e-9803-4463-8fb9-aea24cbd3e32 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:40.114272 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b6e7e-9803-4463-8fb9-aea24cbd3e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:40.114772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b6e7e-9803-4463-8fb9-aea24cbd3e32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:40.115838 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b6e7e-9803-4463-8fb9-aea24cbd3e32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:40.118804 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b6e7e-9803-4463-8fb9-aea24cbd3e32 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:40.171479 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1303/2596] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:58:39 2022] POST /identity/v3/projects => generated 401 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:58:40.176636 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2fd9b-c89f-4741-87ca-7e32dead79de None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:40.195304 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2fd9b-c89f-4741-87ca-7e32dead79de None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:40.315749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf4483-b5f3-4bb3-9c98-5e32de770af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:40.316795 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf4483-b5f3-4bb3-9c98-5e32de770af0 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:40.316795 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf4483-b5f3-4bb3-9c98-5e32de770af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:40.323509 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf4483-b5f3-4bb3-9c98-5e32de770af0 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:40.324328 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf4483-b5f3-4bb3-9c98-5e32de770af0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:40.328745 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf4483-b5f3-4bb3-9c98-5e32de770af0 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:40.379709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1294/2597] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:58:40 2022] POST /identity/v3/users => generated 307 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:58:40.384904 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b6e7e-9803-4463-8fb9-aea24cbd3e32 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:40.400031 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b6e7e-9803-4463-8fb9-aea24cbd3e32 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:40.410921 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18be720-be12-4c36-b3ae-430aee22201b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:40.411628 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18be720-be12-4c36-b3ae-430aee22201b admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:40.414437 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18be720-be12-4c36-b3ae-430aee22201b admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:40.414812 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18be720-be12-4c36-b3ae-430aee22201b admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:40.415543 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18be720-be12-4c36-b3ae-430aee22201b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:40.417510 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18be720-be12-4c36-b3ae-430aee22201b admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:40.451616 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf4483-b5f3-4bb3-9c98-5e32de770af0 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:40.461266 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1304/2598] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:40 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:58:40.466111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf4483-b5f3-4bb3-9c98-5e32de770af0 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:40.481536 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e8e58-e01e-4330-a1e5-2925d1267b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:40.481536 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e8e58-e01e-4330-a1e5-2925d1267b71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:40.481536 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e8e58-e01e-4330-a1e5-2925d1267b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:40.481536 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e8e58-e01e-4330-a1e5-2925d1267b71 admin admin] PATH_INFO: `/v3/projects/69323d1a5d5942ec92f15c3b2c319dab/users/4c612d91525446f9a3fa6c188de80510/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:40.521569 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e8e58-e01e-4330-a1e5-2925d1267b71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69323d1a5d5942ec92f15c3b2c319dab, user_id=4c612d91525446f9a3fa6c188de80510, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:40.527434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e8e58-e01e-4330-a1e5-2925d1267b71 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:40.564103 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1295/2599] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:58:40 2022] PUT /identity/v3/projects/69323d1a5d5942ec92f15c3b2c319dab/users/4c612d91525446f9a3fa6c188de80510/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:40.569131 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18be720-be12-4c36-b3ae-430aee22201b admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:40.583949 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18be720-be12-4c36-b3ae-430aee22201b admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:40.701801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d30fc-8319-4e1c-9024-fd2eb22e7067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:40.702548 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d30fc-8319-4e1c-9024-fd2eb22e7067 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:40.702893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d30fc-8319-4e1c-9024-fd2eb22e7067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:40.703268 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d30fc-8319-4e1c-9024-fd2eb22e7067 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:40.703999 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d30fc-8319-4e1c-9024-fd2eb22e7067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:40.706437 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d30fc-8319-4e1c-9024-fd2eb22e7067 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:40.733311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1305/2600] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:40 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:58:40.734623 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e8e58-e01e-4330-a1e5-2925d1267b71 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:40.760307 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e8e58-e01e-4330-a1e5-2925d1267b71 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:40.765020 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8df2a-dfe4-4eea-bb0d-88fb292a4b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:40.765711 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8df2a-dfe4-4eea-bb0d-88fb292a4b5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:40.766068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8df2a-dfe4-4eea-bb0d-88fb292a4b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:40.766489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8df2a-dfe4-4eea-bb0d-88fb292a4b5e admin admin] PATH_INFO: `/v3/projects/69323d1a5d5942ec92f15c3b2c319dab/users/4c612d91525446f9a3fa6c188de80510/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:40.770637 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8df2a-dfe4-4eea-bb0d-88fb292a4b5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=69323d1a5d5942ec92f15c3b2c319dab, user_id=4c612d91525446f9a3fa6c188de80510, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:40.772754 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8df2a-dfe4-4eea-bb0d-88fb292a4b5e admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:40.814706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a292878-67c6-48d5-8f70-59360bbb0fe4 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:40.815206 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a292878-67c6-48d5-8f70-59360bbb0fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:40.815652 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a292878-67c6-48d5-8f70-59360bbb0fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:40.818745 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1296/2601] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:58:40 2022] PUT /identity/v3/projects/69323d1a5d5942ec92f15c3b2c319dab/users/4c612d91525446f9a3fa6c188de80510/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:40.847899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:40.863189 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-2a292878-67c6-48d5-8f70-59360bbb0fe4 None None] MFA Rules not processed for user `4c612d91525446f9a3fa6c188de80510`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:40.868742 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-2a292878-67c6-48d5-8f70-59360bbb0fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:41.001128 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1306/2602] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:40 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:41.013128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8df2a-dfe4-4eea-bb0d-88fb292a4b5e admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:41.015294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-96f8df2a-dfe4-4eea-bb0d-88fb292a4b5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:41.089187 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:41.145874 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8df2a-dfe4-4eea-bb0d-88fb292a4b5e admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:41.156377 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3daa1e-7517-4b07-ac66-658b5e254a57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:41.157022 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3daa1e-7517-4b07-ac66-658b5e254a57 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:41.157292 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3daa1e-7517-4b07-ac66-658b5e254a57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:41.157645 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3daa1e-7517-4b07-ac66-658b5e254a57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:41.172338 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3daa1e-7517-4b07-ac66-658b5e254a57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:41.173895 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3daa1e-7517-4b07-ac66-658b5e254a57 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:41.199856 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1297/2603] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:41 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:58:41.280427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a292878-67c6-48d5-8f70-59360bbb0fe4 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:41.282156 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-2a292878-67c6-48d5-8f70-59360bbb0fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:41.378044 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3daa1e-7517-4b07-ac66-658b5e254a57 service cinder] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:41.391349 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3daa1e-7517-4b07-ac66-658b5e254a57 service cinder] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:41.395373 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c42a7-0b6a-4890-8a24-5f9b5f681252 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:41.396231 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c42a7-0b6a-4890-8a24-5f9b5f681252 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:41.396532 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c42a7-0b6a-4890-8a24-5f9b5f681252 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:41.396887 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c42a7-0b6a-4890-8a24-5f9b5f681252 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:41.426194 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a292878-67c6-48d5-8f70-59360bbb0fe4 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:41.437696 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ea486-02d3-4d68-b0e7-eec92b2c5360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:41.438359 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ea486-02d3-4d68-b0e7-eec92b2c5360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:41.438639 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ea486-02d3-4d68-b0e7-eec92b2c5360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:41.438910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ea486-02d3-4d68-b0e7-eec92b2c5360 admin admin] PATH_INFO: `/v3/users/6d1a15d59d5d476ca6e0ab5176d310cd` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:41.468880 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ea486-02d3-4d68-b0e7-eec92b2c5360 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d1a15d59d5d476ca6e0ab5176d310cd)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:41.470137 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ea486-02d3-4d68-b0e7-eec92b2c5360 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:41.471902 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c42a7-0b6a-4890-8a24-5f9b5f681252 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:41.474187 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c42a7-0b6a-4890-8a24-5f9b5f681252 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:41.504188 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1298/2604] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:41 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:58:41.552156 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-1a8ea486-02d3-4d68-b0e7-eec92b2c5360 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d1a15d59d5d476ca6e0ab5176d310cd'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:41.568610 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-1a8ea486-02d3-4d68-b0e7-eec92b2c5360 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d1a15d59d5d476ca6e0ab5176d310cd'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:41.584869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-1a8ea486-02d3-4d68-b0e7-eec92b2c5360 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d1a15d59d5d476ca6e0ab5176d310cd'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:41.586244 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-1a8ea486-02d3-4d68-b0e7-eec92b2c5360 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d1a15d59d5d476ca6e0ab5176d310cd'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:41.586841 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-1a8ea486-02d3-4d68-b0e7-eec92b2c5360 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d1a15d59d5d476ca6e0ab5176d310cd'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:41.657532 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1307/2605] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:58:41 2022] DELETE /identity/v3/users/6d1a15d59d5d476ca6e0ab5176d310cd => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:41.956955 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c42a7-0b6a-4890-8a24-5f9b5f681252 service cinder] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:41.983076 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c42a7-0b6a-4890-8a24-5f9b5f681252 service cinder] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:42.093882 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d87a2-fc9b-461b-bb1e-56b5fc8a2f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:42.096551 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d87a2-fc9b-461b-bb1e-56b5fc8a2f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:42.096551 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d87a2-fc9b-461b-bb1e-56b5fc8a2f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:42.096551 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d87a2-fc9b-461b-bb1e-56b5fc8a2f86 admin admin] PATH_INFO: `/v3/projects/885b952a63b34e1ba97215831cffc58e` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:42.097939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d87a2-fc9b-461b-bb1e-56b5fc8a2f86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=885b952a63b34e1ba97215831cffc58e)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:42.101597 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d87a2-fc9b-461b-bb1e-56b5fc8a2f86 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:42.173373 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ef3d87a2-fc9b-461b-bb1e-56b5fc8a2f86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '885b952a63b34e1ba97215831cffc58e', 'request_id': 'req-ef3d87a2-fc9b-461b-bb1e-56b5fc8a2f86'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:42.194638 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ef3d87a2-fc9b-461b-bb1e-56b5fc8a2f86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '885b952a63b34e1ba97215831cffc58e', 'request_id': 'req-ef3d87a2-fc9b-461b-bb1e-56b5fc8a2f86'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:42.209594 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ef3d87a2-fc9b-461b-bb1e-56b5fc8a2f86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '885b952a63b34e1ba97215831cffc58e', 'request_id': 'req-ef3d87a2-fc9b-461b-bb1e-56b5fc8a2f86'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:42.222622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:42.231813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ef3d87a2-fc9b-461b-bb1e-56b5fc8a2f86 admin admin] The token cache is being invalidate because project 885b952a63b34e1ba97215831cffc58e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:58:42.232116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ef3d87a2-fc9b-461b-bb1e-56b5fc8a2f86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:42.233961 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ef3d87a2-fc9b-461b-bb1e-56b5fc8a2f86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:42.239596 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1299/2606] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:58:41 2022] DELETE /identity/v3/projects/885b952a63b34e1ba97215831cffc58e => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:42.240801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb3888-45f5-427f-9fa5-75a4934f1ea8 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:42.242556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb3888-45f5-427f-9fa5-75a4934f1ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:42.242556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb3888-45f5-427f-9fa5-75a4934f1ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:42.253292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:42.277918 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-75cb3888-45f5-427f-9fa5-75a4934f1ea8 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:42.280556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-75cb3888-45f5-427f-9fa5-75a4934f1ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:42.365936 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1308/2607] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:42.373710 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aee057-dbef-41af-93d0-7379f2b57413 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:42.373912 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aee057-dbef-41af-93d0-7379f2b57413 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:42.374177 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aee057-dbef-41af-93d0-7379f2b57413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:42.396593 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-09aee057-dbef-41af-93d0-7379f2b57413 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:42.398654 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-09aee057-dbef-41af-93d0-7379f2b57413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:42.409427 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1300/2608] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:42.413933 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb3888-45f5-427f-9fa5-75a4934f1ea8 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:42.423958 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb3888-45f5-427f-9fa5-75a4934f1ea8 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:42.428761 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dafd55a-0580-4a8e-bcb6-c60f1b93ecd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:42.429935 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafd55a-0580-4a8e-bcb6-c60f1b93ecd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:42.430477 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafd55a-0580-4a8e-bcb6-c60f1b93ecd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:42.430806 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafd55a-0580-4a8e-bcb6-c60f1b93ecd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:42.431482 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dafd55a-0580-4a8e-bcb6-c60f1b93ecd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:58:42.432018 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dafd55a-0580-4a8e-bcb6-c60f1b93ecd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:42.433829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dafd55a-0580-4a8e-bcb6-c60f1b93ecd1 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:42.446792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1309/2609] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:58:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:58:42.451368 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aee057-dbef-41af-93d0-7379f2b57413 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:42.461822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aee057-dbef-41af-93d0-7379f2b57413 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:42.466780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a1492-68f5-4c17-8aa0-e72125fbf174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:42.467418 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a1492-68f5-4c17-8aa0-e72125fbf174 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:42.467702 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a1492-68f5-4c17-8aa0-e72125fbf174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:42.467979 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a1492-68f5-4c17-8aa0-e72125fbf174 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:42.468660 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a1492-68f5-4c17-8aa0-e72125fbf174 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:42.470821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a1492-68f5-4c17-8aa0-e72125fbf174 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:42.514065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1301/2610] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:58:42 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:58:42.518437 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dafd55a-0580-4a8e-bcb6-c60f1b93ecd1 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:42.529567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dafd55a-0580-4a8e-bcb6-c60f1b93ecd1 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:42.588103 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9ebe7-62f4-4552-849b-92bdd01d5883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:42.588992 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9ebe7-62f4-4552-849b-92bdd01d5883 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:42.589273 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9ebe7-62f4-4552-849b-92bdd01d5883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:42.589543 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9ebe7-62f4-4552-849b-92bdd01d5883 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:42.590181 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9ebe7-62f4-4552-849b-92bdd01d5883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:42.592148 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9ebe7-62f4-4552-849b-92bdd01d5883 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:42.618658 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1310/2611] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:58:42 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:58:42.622758 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a1492-68f5-4c17-8aa0-e72125fbf174 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:42.633390 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a1492-68f5-4c17-8aa0-e72125fbf174 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:42.638180 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247db14a-56d2-4cc7-b1f5-7bf39875312a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:42.639052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247db14a-56d2-4cc7-b1f5-7bf39875312a admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:42.639178 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247db14a-56d2-4cc7-b1f5-7bf39875312a admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:42.639622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247db14a-56d2-4cc7-b1f5-7bf39875312a admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:42.640268 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247db14a-56d2-4cc7-b1f5-7bf39875312a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:42.641875 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247db14a-56d2-4cc7-b1f5-7bf39875312a admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:42.657103 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1302/2612] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:58:42.658556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9ebe7-62f4-4552-849b-92bdd01d5883 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:42.675501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9ebe7-62f4-4552-849b-92bdd01d5883 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:42.681530 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c843233-f2bb-4232-b209-34aa77a944e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:42.682147 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c843233-f2bb-4232-b209-34aa77a944e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:42.682421 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c843233-f2bb-4232-b209-34aa77a944e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:42.682690 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c843233-f2bb-4232-b209-34aa77a944e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:42.683366 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c843233-f2bb-4232-b209-34aa77a944e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:42.684944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c843233-f2bb-4232-b209-34aa77a944e7 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:42.699775 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1311/2613] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:58:42.701151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247db14a-56d2-4cc7-b1f5-7bf39875312a admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:42.716695 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247db14a-56d2-4cc7-b1f5-7bf39875312a admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:42.723517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894d259-72a1-4147-a83b-5466a81466de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:42.724194 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894d259-72a1-4147-a83b-5466a81466de admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:42.724478 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894d259-72a1-4147-a83b-5466a81466de admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:42.724756 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894d259-72a1-4147-a83b-5466a81466de admin admin] PATH_INFO: `/v3/projects/6b67e536fd414591b859c1a89808e1a6/users/26826ddf92124c95835a5356e13040bf/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:42.741202 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894d259-72a1-4147-a83b-5466a81466de admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b67e536fd414591b859c1a89808e1a6, user_id=26826ddf92124c95835a5356e13040bf, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:42.744273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894d259-72a1-4147-a83b-5466a81466de admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:42.758856 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1303/2614] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:58:42 2022] PUT /identity/v3/projects/6b67e536fd414591b859c1a89808e1a6/users/26826ddf92124c95835a5356e13040bf/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:42.764053 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683bd6c-0ebb-42d6-9b5d-594327096bf7 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:42.764333 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683bd6c-0ebb-42d6-9b5d-594327096bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:42.764675 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683bd6c-0ebb-42d6-9b5d-594327096bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:42.785192 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-f683bd6c-0ebb-42d6-9b5d-594327096bf7 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:42.786438 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f683bd6c-0ebb-42d6-9b5d-594327096bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:42.858816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1312/2615] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:42.869219 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894d259-72a1-4147-a83b-5466a81466de admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:42.870934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-d894d259-72a1-4147-a83b-5466a81466de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:42.935615 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894d259-72a1-4147-a83b-5466a81466de admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:42.939518 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef9b4c-bf43-496c-8018-bc2c6f10a696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:42.940136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef9b4c-bf43-496c-8018-bc2c6f10a696 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:42.940417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef9b4c-bf43-496c-8018-bc2c6f10a696 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:42.940694 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef9b4c-bf43-496c-8018-bc2c6f10a696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:42.950604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ef9b4c-bf43-496c-8018-bc2c6f10a696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:42.952073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ef9b4c-bf43-496c-8018-bc2c6f10a696 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:42.962592 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1304/2616] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:58:48.675266 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683bd6c-0ebb-42d6-9b5d-594327096bf7 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:48.676862 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f683bd6c-0ebb-42d6-9b5d-594327096bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:48.701341 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683bd6c-0ebb-42d6-9b5d-594327096bf7 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:48.706674 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9ea86-ec22-4175-ba15-b27c948abf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:48.707474 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9ea86-ec22-4175-ba15-b27c948abf05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:48.707810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9ea86-ec22-4175-ba15-b27c948abf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:48.708152 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9ea86-ec22-4175-ba15-b27c948abf05 admin admin] PATH_INFO: `/v3/users/0b7e62a06c3b46ce9b619c277c5629e1` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:48.709390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9ea86-ec22-4175-ba15-b27c948abf05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b7e62a06c3b46ce9b619c277c5629e1)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:48.712639 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9ea86-ec22-4175-ba15-b27c948abf05 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:48.720512 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:48.780636 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c4d9ea86-ec22-4175-ba15-b27c948abf05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b7e62a06c3b46ce9b619c277c5629e1'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:48.792493 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c4d9ea86-ec22-4175-ba15-b27c948abf05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b7e62a06c3b46ce9b619c277c5629e1'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:48.803114 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c4d9ea86-ec22-4175-ba15-b27c948abf05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b7e62a06c3b46ce9b619c277c5629e1'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:48.805361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c4d9ea86-ec22-4175-ba15-b27c948abf05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b7e62a06c3b46ce9b619c277c5629e1'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:48.806221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c4d9ea86-ec22-4175-ba15-b27c948abf05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b7e62a06c3b46ce9b619c277c5629e1'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:48.812288 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa4f53-851a-459f-8d12-d83c86d73019 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:48.812788 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa4f53-851a-459f-8d12-d83c86d73019 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:48.813089 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa4f53-851a-459f-8d12-d83c86d73019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:48.840714 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1313/2617] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:58:48 2022] DELETE /identity/v3/users/0b7e62a06c3b46ce9b619c277c5629e1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:48.845254 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409fe452-0f3f-4e2a-ba8f-acd7dfe7231f None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:48.845632 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409fe452-0f3f-4e2a-ba8f-acd7dfe7231f None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:48.846072 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409fe452-0f3f-4e2a-ba8f-acd7dfe7231f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:48.858187 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-bffa4f53-851a-459f-8d12-d83c86d73019 None None] MFA Rules not processed for user `26826ddf92124c95835a5356e13040bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:48.859586 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-bffa4f53-851a-459f-8d12-d83c86d73019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:48.875101 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-409fe452-0f3f-4e2a-ba8f-acd7dfe7231f None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:48.875101 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-409fe452-0f3f-4e2a-ba8f-acd7dfe7231f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:48.942695 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1305/2618] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:58:48.942975 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1314/2619] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:58:48.948993 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffa4f53-851a-459f-8d12-d83c86d73019 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:48.949744 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409fe452-0f3f-4e2a-ba8f-acd7dfe7231f None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:48.950371 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-bffa4f53-851a-459f-8d12-d83c86d73019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:48.951242 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-409fe452-0f3f-4e2a-ba8f-acd7dfe7231f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:48.985001 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:49.026103 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffa4f53-851a-459f-8d12-d83c86d73019 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:49.029853 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890bae0-76d2-4e19-8eec-7776b43c2ec7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:49.030468 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890bae0-76d2-4e19-8eec-7776b43c2ec7 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:49.030747 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890bae0-76d2-4e19-8eec-7776b43c2ec7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:49.031021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890bae0-76d2-4e19-8eec-7776b43c2ec7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:49.041125 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f890bae0-76d2-4e19-8eec-7776b43c2ec7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:49.041499 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409fe452-0f3f-4e2a-ba8f-acd7dfe7231f None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:49.042577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f890bae0-76d2-4e19-8eec-7776b43c2ec7 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:49.047343 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138e1c2-7600-4d70-b82d-92f3b2b9d202 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:49.047968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138e1c2-7600-4d70-b82d-92f3b2b9d202 service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:49.048245 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138e1c2-7600-4d70-b82d-92f3b2b9d202 service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:49.048513 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138e1c2-7600-4d70-b82d-92f3b2b9d202 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:49.052447 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1306/2620] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:58:49.057238 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7138e1c2-7600-4d70-b82d-92f3b2b9d202 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:49.059159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7138e1c2-7600-4d70-b82d-92f3b2b9d202 service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:49.068563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1315/2621] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:58:49.352079 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890bae0-76d2-4e19-8eec-7776b43c2ec7 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:49.353557 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-f890bae0-76d2-4e19-8eec-7776b43c2ec7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:49.377329 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890bae0-76d2-4e19-8eec-7776b43c2ec7 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:49.384063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19bb3ed-8f5f-4b7b-9254-0852fe6e9035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:49.384710 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bb3ed-8f5f-4b7b-9254-0852fe6e9035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:49.384987 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bb3ed-8f5f-4b7b-9254-0852fe6e9035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:49.385399 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bb3ed-8f5f-4b7b-9254-0852fe6e9035 admin admin] PATH_INFO: `/v3/projects/e3e7327226f34401b53130eb712049fc` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:49.386571 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19bb3ed-8f5f-4b7b-9254-0852fe6e9035 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3e7327226f34401b53130eb712049fc)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:49.388495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19bb3ed-8f5f-4b7b-9254-0852fe6e9035 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:49.454081 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b19bb3ed-8f5f-4b7b-9254-0852fe6e9035 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3e7327226f34401b53130eb712049fc', 'request_id': 'req-b19bb3ed-8f5f-4b7b-9254-0852fe6e9035'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:49.465119 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b19bb3ed-8f5f-4b7b-9254-0852fe6e9035 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3e7327226f34401b53130eb712049fc', 'request_id': 'req-b19bb3ed-8f5f-4b7b-9254-0852fe6e9035'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:49.474587 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b19bb3ed-8f5f-4b7b-9254-0852fe6e9035 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3e7327226f34401b53130eb712049fc', 'request_id': 'req-b19bb3ed-8f5f-4b7b-9254-0852fe6e9035'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:49.490734 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b19bb3ed-8f5f-4b7b-9254-0852fe6e9035 admin admin] The token cache is being invalidate because project e3e7327226f34401b53130eb712049fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:58:49.491258 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b19bb3ed-8f5f-4b7b-9254-0852fe6e9035 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:49.492024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b19bb3ed-8f5f-4b7b-9254-0852fe6e9035 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:49.496347 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1307/2622] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:58:49 2022] DELETE /identity/v3/projects/e3e7327226f34401b53130eb712049fc => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:58:49.500353 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138e1c2-7600-4d70-b82d-92f3b2b9d202 service nova] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:49.514861 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138e1c2-7600-4d70-b82d-92f3b2b9d202 service nova] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:49.523692 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:49.614938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427379e0-22d4-4668-bd02-f5e87b2eb8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:49.615746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427379e0-22d4-4668-bd02-f5e87b2eb8dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:49.615940 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427379e0-22d4-4668-bd02-f5e87b2eb8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:49.616246 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427379e0-22d4-4668-bd02-f5e87b2eb8dd admin admin] PATH_INFO: `/v3/users/4c612d91525446f9a3fa6c188de80510` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:49.621055 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427379e0-22d4-4668-bd02-f5e87b2eb8dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c612d91525446f9a3fa6c188de80510)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:49.623481 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427379e0-22d4-4668-bd02-f5e87b2eb8dd admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:49.693337 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-427379e0-22d4-4668-bd02-f5e87b2eb8dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c612d91525446f9a3fa6c188de80510'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:49.705127 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-427379e0-22d4-4668-bd02-f5e87b2eb8dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c612d91525446f9a3fa6c188de80510'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:49.716056 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-427379e0-22d4-4668-bd02-f5e87b2eb8dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c612d91525446f9a3fa6c188de80510'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:49.716812 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-427379e0-22d4-4668-bd02-f5e87b2eb8dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c612d91525446f9a3fa6c188de80510'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:49.717413 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-427379e0-22d4-4668-bd02-f5e87b2eb8dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c612d91525446f9a3fa6c188de80510'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:49.739421 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1316/2623] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:58:49 2022] DELETE /identity/v3/users/4c612d91525446f9a3fa6c188de80510 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:50.311315 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19bb3ed-8f5f-4b7b-9254-0852fe6e9035 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:50.313358 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b19bb3ed-8f5f-4b7b-9254-0852fe6e9035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:50.330072 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19bb3ed-8f5f-4b7b-9254-0852fe6e9035 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:50.411662 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8928019a-3929-4412-83cb-586b7ae213ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:50.412299 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928019a-3929-4412-83cb-586b7ae213ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:50.412578 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928019a-3929-4412-83cb-586b7ae213ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:50.412874 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928019a-3929-4412-83cb-586b7ae213ee admin admin] PATH_INFO: `/v3/projects/69323d1a5d5942ec92f15c3b2c319dab` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:50.414062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8928019a-3929-4412-83cb-586b7ae213ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=69323d1a5d5942ec92f15c3b2c319dab)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:50.416006 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8928019a-3929-4412-83cb-586b7ae213ee admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:50.489769 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-8928019a-3929-4412-83cb-586b7ae213ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69323d1a5d5942ec92f15c3b2c319dab', 'request_id': 'req-8928019a-3929-4412-83cb-586b7ae213ee'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:50.495462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:50.502389 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-8928019a-3929-4412-83cb-586b7ae213ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69323d1a5d5942ec92f15c3b2c319dab', 'request_id': 'req-8928019a-3929-4412-83cb-586b7ae213ee'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:50.513393 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-8928019a-3929-4412-83cb-586b7ae213ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69323d1a5d5942ec92f15c3b2c319dab', 'request_id': 'req-8928019a-3929-4412-83cb-586b7ae213ee'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:50.526869 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-8928019a-3929-4412-83cb-586b7ae213ee admin admin] The token cache is being invalidate because project 69323d1a5d5942ec92f15c3b2c319dab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:58:50.527820 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-8928019a-3929-4412-83cb-586b7ae213ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:50.528163 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-8928019a-3929-4412-83cb-586b7ae213ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:50.530187 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1308/2624] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:58:50 2022] DELETE /identity/v3/projects/69323d1a5d5942ec92f15c3b2c319dab => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:50.536271 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468b4a4-b0ae-4f66-bdaf-a0e95d554b47 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:50.537630 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468b4a4-b0ae-4f66-bdaf-a0e95d554b47 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:50.538748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468b4a4-b0ae-4f66-bdaf-a0e95d554b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:50.569286 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-e468b4a4-b0ae-4f66-bdaf-a0e95d554b47 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:50.571652 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e468b4a4-b0ae-4f66-bdaf-a0e95d554b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:50.685107 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1317/2625] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:50.692915 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a7c61-8c51-482f-87db-ba78160ef19c None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:50.693203 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a7c61-8c51-482f-87db-ba78160ef19c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:50.693469 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a7c61-8c51-482f-87db-ba78160ef19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:50.697165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8cce8-2e8b-4f90-9a12-434781beb877 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:50.697706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8cce8-2e8b-4f90-9a12-434781beb877 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:50.697777 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8cce8-2e8b-4f90-9a12-434781beb877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:50.718057 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-6dc8cce8-2e8b-4f90-9a12-434781beb877 None None] MFA Rules not processed for user `26826ddf92124c95835a5356e13040bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:50.719596 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-6dc8cce8-2e8b-4f90-9a12-434781beb877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:50.722169 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-785a7c61-8c51-482f-87db-ba78160ef19c None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:50.723365 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-785a7c61-8c51-482f-87db-ba78160ef19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:50.732466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1309/2626] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:50.736916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a7c61-8c51-482f-87db-ba78160ef19c None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:50.751146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a7c61-8c51-482f-87db-ba78160ef19c None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:50.756355 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:50.757812 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25cad9-2b49-4311-8f0b-7a7a6043a62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:50.758607 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25cad9-2b49-4311-8f0b-7a7a6043a62b admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:50.759015 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25cad9-2b49-4311-8f0b-7a7a6043a62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:50.759331 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25cad9-2b49-4311-8f0b-7a7a6043a62b admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:50.760046 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25cad9-2b49-4311-8f0b-7a7a6043a62b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:58:50.760382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25cad9-2b49-4311-8f0b-7a7a6043a62b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:50.762039 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25cad9-2b49-4311-8f0b-7a7a6043a62b admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:50.779934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1310/2627] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:58:50.784100 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25cad9-2b49-4311-8f0b-7a7a6043a62b admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:50.795610 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25cad9-2b49-4311-8f0b-7a7a6043a62b admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:50.801109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eefc72-72a5-4524-83f3-fd52a44fa8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:50.803165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eefc72-72a5-4524-83f3-fd52a44fa8b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:50.803165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eefc72-72a5-4524-83f3-fd52a44fa8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:50.803165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eefc72-72a5-4524-83f3-fd52a44fa8b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:50.803165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eefc72-72a5-4524-83f3-fd52a44fa8b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:50.805639 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eefc72-72a5-4524-83f3-fd52a44fa8b2 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:50.808190 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1318/2628] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:50.848863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1311/2629] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:58:50 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:58:50.853657 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8cce8-2e8b-4f90-9a12-434781beb877 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:50.864769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8cce8-2e8b-4f90-9a12-434781beb877 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:50.956627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4c2df-8d0e-457c-8050-257a1577e467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:50.957257 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4c2df-8d0e-457c-8050-257a1577e467 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:50.957540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4c2df-8d0e-457c-8050-257a1577e467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:50.957822 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4c2df-8d0e-457c-8050-257a1577e467 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:50.958586 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4c2df-8d0e-457c-8050-257a1577e467 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:50.961474 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4c2df-8d0e-457c-8050-257a1577e467 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:50.987276 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1319/2630] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:58:50 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:58:50.991213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eefc72-72a5-4524-83f3-fd52a44fa8b2 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:51.006169 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eefc72-72a5-4524-83f3-fd52a44fa8b2 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:51.013173 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac729d38-197e-48a4-b643-35789c871472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:51.014062 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac729d38-197e-48a4-b643-35789c871472 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:51.014542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac729d38-197e-48a4-b643-35789c871472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:51.015083 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac729d38-197e-48a4-b643-35789c871472 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:51.015932 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac729d38-197e-48a4-b643-35789c871472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:51.017852 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac729d38-197e-48a4-b643-35789c871472 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:51.034954 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4c2df-8d0e-457c-8050-257a1577e467 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:51.035583 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1312/2631] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:58:51.060904 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4c2df-8d0e-457c-8050-257a1577e467 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:51.072550 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ccb1c-5686-4251-9da0-d7a859a3778a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:51.073191 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ccb1c-5686-4251-9da0-d7a859a3778a admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:51.073619 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ccb1c-5686-4251-9da0-d7a859a3778a admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:51.074034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ccb1c-5686-4251-9da0-d7a859a3778a admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:51.074704 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ccb1c-5686-4251-9da0-d7a859a3778a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:51.082778 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ccb1c-5686-4251-9da0-d7a859a3778a admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:51.095850 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1320/2632] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 00:58:51.103799 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac729d38-197e-48a4-b643-35789c871472 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:51.120901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac729d38-197e-48a4-b643-35789c871472 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:51.131391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8cdaa2-235c-4126-85ff-d79a0a525fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:51.133574 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cdaa2-235c-4126-85ff-d79a0a525fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:51.133964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cdaa2-235c-4126-85ff-d79a0a525fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:51.134249 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cdaa2-235c-4126-85ff-d79a0a525fc2 admin admin] PATH_INFO: `/v3/projects/bd0c7b2d6c6e4ce487bc08422bcde3e9/users/46728ca0b6c347078c0e6d1862ff94ce/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:51.212193 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ccb1c-5686-4251-9da0-d7a859a3778a admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:51.214104 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-bf2ccb1c-5686-4251-9da0-d7a859a3778a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:51.302078 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8cdaa2-235c-4126-85ff-d79a0a525fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd0c7b2d6c6e4ce487bc08422bcde3e9, user_id=46728ca0b6c347078c0e6d1862ff94ce, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:51.305444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8cdaa2-235c-4126-85ff-d79a0a525fc2 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:51.314720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ccb1c-5686-4251-9da0-d7a859a3778a admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:51.332519 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1313/2633] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:58:51 2022] PUT /identity/v3/projects/bd0c7b2d6c6e4ce487bc08422bcde3e9/users/46728ca0b6c347078c0e6d1862ff94ce/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:51.334204 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86baabd3-1a73-4dbc-b525-196270a68094 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:51.334498 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86baabd3-1a73-4dbc-b525-196270a68094 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:51.336019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86baabd3-1a73-4dbc-b525-196270a68094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:51.372295 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-86baabd3-1a73-4dbc-b525-196270a68094 None None] MFA Rules not processed for user `46728ca0b6c347078c0e6d1862ff94ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:51.373134 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-86baabd3-1a73-4dbc-b525-196270a68094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:51.408212 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd39763-4129-4da0-ac3d-f6e561eef025 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:51.409222 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd39763-4129-4da0-ac3d-f6e561eef025 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:51.409320 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd39763-4129-4da0-ac3d-f6e561eef025 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:51.409663 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd39763-4129-4da0-ac3d-f6e561eef025 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:51.415684 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-8cd39763-4129-4da0-ac3d-f6e561eef025 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:51.437810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd39763-4129-4da0-ac3d-f6e561eef025 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:51.437810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd39763-4129-4da0-ac3d-f6e561eef025 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:51.467869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1321/2634] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:58:51.499304 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1314/2635] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:51.508395 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd39763-4129-4da0-ac3d-f6e561eef025 service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:51.542687 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:51.665878 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd39763-4129-4da0-ac3d-f6e561eef025 service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:51.671098 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e194e-84a7-450f-a208-734e497cabb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:51.671729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e194e-84a7-450f-a208-734e497cabb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:51.672000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e194e-84a7-450f-a208-734e497cabb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:51.672421 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e194e-84a7-450f-a208-734e497cabb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:51.673072 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589e194e-84a7-450f-a208-734e497cabb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:51.675171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589e194e-84a7-450f-a208-734e497cabb0 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:51.722278 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1322/2636] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:58:51 2022] POST /identity/v3/projects => generated 383 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:58:51.728640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86baabd3-1a73-4dbc-b525-196270a68094 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:51.750453 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86baabd3-1a73-4dbc-b525-196270a68094 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:51.855419 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:51.863209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b590607-8308-47ae-bf29-e31c5dc1da13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:51.863916 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b590607-8308-47ae-bf29-e31c5dc1da13 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:51.868291 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b590607-8308-47ae-bf29-e31c5dc1da13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:51.868631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b590607-8308-47ae-bf29-e31c5dc1da13 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:51.869258 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b590607-8308-47ae-bf29-e31c5dc1da13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:51.874355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b590607-8308-47ae-bf29-e31c5dc1da13 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:51.909834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1315/2637] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:58:51 2022] POST /identity/v3/users => generated 298 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:58:51.914334 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e194e-84a7-450f-a208-734e497cabb0 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:51.931630 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e194e-84a7-450f-a208-734e497cabb0 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:51.945450 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7e102-dd21-4e8f-a042-e95fd113b5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:51.946281 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7e102-dd21-4e8f-a042-e95fd113b5be admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:51.946564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7e102-dd21-4e8f-a042-e95fd113b5be admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:51.946834 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7e102-dd21-4e8f-a042-e95fd113b5be admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:51.947539 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7e102-dd21-4e8f-a042-e95fd113b5be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:51.951080 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7e102-dd21-4e8f-a042-e95fd113b5be admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:51.976805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1323/2638] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:58:51.981317 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b590607-8308-47ae-bf29-e31c5dc1da13 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:52.000847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b590607-8308-47ae-bf29-e31c5dc1da13 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:52.020236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f8bfa-f170-4899-9f38-16f7bcff75a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:52.020236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f8bfa-f170-4899-9f38-16f7bcff75a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:52.020236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f8bfa-f170-4899-9f38-16f7bcff75a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:52.020236 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f8bfa-f170-4899-9f38-16f7bcff75a1 admin admin] PATH_INFO: `/v3/projects/489f153f2f154c3b9811b08d2e3cbb80/users/75fad4bb2b5a44779e527d42617a02d3/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:52.032444 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7e102-dd21-4e8f-a042-e95fd113b5be admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:52.035024 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-58e7e102-dd21-4e8f-a042-e95fd113b5be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:52.068556 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481f8bfa-f170-4899-9f38-16f7bcff75a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=489f153f2f154c3b9811b08d2e3cbb80, user_id=75fad4bb2b5a44779e527d42617a02d3, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:52.071864 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481f8bfa-f170-4899-9f38-16f7bcff75a1 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:52.106428 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1316/2639] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:58:51 2022] PUT /identity/v3/projects/489f153f2f154c3b9811b08d2e3cbb80/users/75fad4bb2b5a44779e527d42617a02d3/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:52.111294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f8bfa-f170-4899-9f38-16f7bcff75a1 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:52.202955 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7e102-dd21-4e8f-a042-e95fd113b5be admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:52.238418 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f8bfa-f170-4899-9f38-16f7bcff75a1 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:52.246524 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0315f-97ef-4782-b5a4-c329194c5390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:52.247626 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0315f-97ef-4782-b5a4-c329194c5390 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:52.248085 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0315f-97ef-4782-b5a4-c329194c5390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:52.248656 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0315f-97ef-4782-b5a4-c329194c5390 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:52.250109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0315f-97ef-4782-b5a4-c329194c5390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:52.252636 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0315f-97ef-4782-b5a4-c329194c5390 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:52.272240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1317/2640] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:58:52.274252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0315f-97ef-4782-b5a4-c329194c5390 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:52.302908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0315f-97ef-4782-b5a4-c329194c5390 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:52.311995 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0297f-8e96-4fab-9f17-b4055697a3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:52.313344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0297f-8e96-4fab-9f17-b4055697a3f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:52.313829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0297f-8e96-4fab-9f17-b4055697a3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:52.314632 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0297f-8e96-4fab-9f17-b4055697a3f7 admin admin] PATH_INFO: `/v3/projects/489f153f2f154c3b9811b08d2e3cbb80/users/75fad4bb2b5a44779e527d42617a02d3/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:52.318058 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0297f-8e96-4fab-9f17-b4055697a3f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=489f153f2f154c3b9811b08d2e3cbb80, user_id=75fad4bb2b5a44779e527d42617a02d3, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:52.322383 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0297f-8e96-4fab-9f17-b4055697a3f7 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:52.350292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76391f9-ddfd-4f20-a1e4-2b7a97b6460f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdacdcb2451742938164fdd44cdbaebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:52.351737 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76391f9-ddfd-4f20-a1e4-2b7a97b6460f service placement] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:52.352022 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76391f9-ddfd-4f20-a1e4-2b7a97b6460f service placement] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:52.352298 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76391f9-ddfd-4f20-a1e4-2b7a97b6460f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:52.367636 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1318/2641] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:58:52 2022] PUT /identity/v3/projects/489f153f2f154c3b9811b08d2e3cbb80/users/75fad4bb2b5a44779e527d42617a02d3/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:52.369728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18bbf2-97fc-4b3a-a705-c5bd0203db31 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:52.370961 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b76391f9-ddfd-4f20-a1e4-2b7a97b6460f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:52.372556 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18bbf2-97fc-4b3a-a705-c5bd0203db31 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:52.372890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18bbf2-97fc-4b3a-a705-c5bd0203db31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:52.418398 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76391f9-ddfd-4f20-a1e4-2b7a97b6460f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:52.420117 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76391f9-ddfd-4f20-a1e4-2b7a97b6460f service placement] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:52.441439 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1324/2642] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:58:52.486945 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-5e18bbf2-97fc-4b3a-a705-c5bd0203db31 None None] MFA Rules not processed for user `75fad4bb2b5a44779e527d42617a02d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:52.488202 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-5e18bbf2-97fc-4b3a-a705-c5bd0203db31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:52.618525 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1319/2643] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:52 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:52.630574 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76391f9-ddfd-4f20-a1e4-2b7a97b6460f service placement] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:52.632285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b76391f9-ddfd-4f20-a1e4-2b7a97b6460f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:52.654022 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:52.713763 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76391f9-ddfd-4f20-a1e4-2b7a97b6460f service placement] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:52.779804 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b71dd8-c782-4eb4-836c-e52e58f587e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:52.781357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b71dd8-c782-4eb4-836c-e52e58f587e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:52.781357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b71dd8-c782-4eb4-836c-e52e58f587e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:52.781357 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b71dd8-c782-4eb4-836c-e52e58f587e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:52.795223 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b71dd8-c782-4eb4-836c-e52e58f587e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:52.796856 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b71dd8-c782-4eb4-836c-e52e58f587e1 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:52.822176 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1325/2644] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:52 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:58:52.955882 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18bbf2-97fc-4b3a-a705-c5bd0203db31 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:52.974437 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18bbf2-97fc-4b3a-a705-c5bd0203db31 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:52.980231 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c476c64-782c-4dd2-a215-b8bcb3158e83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:52.981373 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c476c64-782c-4dd2-a215-b8bcb3158e83 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:52.981654 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c476c64-782c-4dd2-a215-b8bcb3158e83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:52.981924 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c476c64-782c-4dd2-a215-b8bcb3158e83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:53.018775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:53.089543 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c476c64-782c-4dd2-a215-b8bcb3158e83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:53.091063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c476c64-782c-4dd2-a215-b8bcb3158e83 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:53.095935 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316a7e9-f82d-4e3a-8e7c-c5a50b011790 None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:53.096171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316a7e9-f82d-4e3a-8e7c-c5a50b011790 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:53.096445 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316a7e9-f82d-4e3a-8e7c-c5a50b011790 None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:53.097936 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1326/2645] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 31 00:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:58:53.108589 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ed20c-b535-4f79-b110-5667c3c4bcfb None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:53.108936 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ed20c-b535-4f79-b110-5667c3c4bcfb None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:53.109216 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ed20c-b535-4f79-b110-5667c3c4bcfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:53.127023 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1320/2646] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:58:53.188747 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-a49ed20c-b535-4f79-b110-5667c3c4bcfb None None] MFA Rules not processed for user `3aa53ae5a30146f69d7d6eefe4290dba`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:53.190640 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a49ed20c-b535-4f79-b110-5667c3c4bcfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:53.285613 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1327/2647] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 31 00:58:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:53.299302 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c476c64-782c-4dd2-a215-b8bcb3158e83 service cinder] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:53.300809 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-9c476c64-782c-4dd2-a215-b8bcb3158e83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:53.399014 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c476c64-782c-4dd2-a215-b8bcb3158e83 service cinder] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:53.402945 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0589d2-05cb-44ab-af1d-27e85579c146 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55714a21c1cb4f2b949603230243ed02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:53.403584 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0589d2-05cb-44ab-af1d-27e85579c146 service glance] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:53.403857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0589d2-05cb-44ab-af1d-27e85579c146 service glance] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:53.404124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0589d2-05cb-44ab-af1d-27e85579c146 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:53.415665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0589d2-05cb-44ab-af1d-27e85579c146 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:53.417085 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0589d2-05cb-44ab-af1d-27e85579c146 service glance] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:53.428512 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1321/2648] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:58:53.518478 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670dc96-ca84-4392-a197-6c1c565b7e6a None None] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:53.518923 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670dc96-ca84-4392-a197-6c1c565b7e6a None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:53.519403 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670dc96-ca84-4392-a197-6c1c565b7e6a None None] PATH_INFO: `/` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:53.523628 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1328/2649] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 31 00:58:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 00:58:53.532316 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faff9fc-26d5-4388-8f17-01bd67936bb3 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:53.532604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faff9fc-26d5-4388-8f17-01bd67936bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:53.532879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faff9fc-26d5-4388-8f17-01bd67936bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:53.586929 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-8faff9fc-26d5-4388-8f17-01bd67936bb3 None None] MFA Rules not processed for user `05c0d9554cf94c41a2cd709cab378e53`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:53.588324 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-8faff9fc-26d5-4388-8f17-01bd67936bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:53.648999 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1322/2650] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 31 00:58:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:58:53.661711 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670dc96-ca84-4392-a197-6c1c565b7e6a None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:53.663817 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-1670dc96-ca84-4392-a197-6c1c565b7e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:53.681294 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670dc96-ca84-4392-a197-6c1c565b7e6a None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:53.685423 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca97f2-1a21-4330-898e-1319f503e746 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:53.686044 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca97f2-1a21-4330-898e-1319f503e746 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:53.686326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca97f2-1a21-4330-898e-1319f503e746 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:53.686600 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca97f2-1a21-4330-898e-1319f503e746 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:53.695522 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca97f2-1a21-4330-898e-1319f503e746 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:53.697087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca97f2-1a21-4330-898e-1319f503e746 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:53.706528 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1329/2651] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:58:54.815892 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431fb4f-deee-4d88-8231-cc3ff7b40c36 None None] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:54.816739 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431fb4f-deee-4d88-8231-cc3ff7b40c36 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:54.817008 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431fb4f-deee-4d88-8231-cc3ff7b40c36 None None] PATH_INFO: `/` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:54.818359 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1323/2652] 10.222.0.67 () {54 vars in 921 bytes} [Wed Aug 31 00:58:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 00:58:54.825914 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310ad16-b58d-4acc-9479-444cf56437c6 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:54.826787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310ad16-b58d-4acc-9479-444cf56437c6 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:54.826787 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310ad16-b58d-4acc-9479-444cf56437c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:54.849920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-b310ad16-b58d-4acc-9479-444cf56437c6 None None] MFA Rules not processed for user `3aa53ae5a30146f69d7d6eefe4290dba`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:54.852609 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b310ad16-b58d-4acc-9479-444cf56437c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:54.864847 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1330/2653] 10.222.0.67 () {58 vars in 1018 bytes} [Wed Aug 31 00:58:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:54.877627 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c431fb4f-deee-4d88-8231-cc3ff7b40c36 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:54.890213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c431fb4f-deee-4d88-8231-cc3ff7b40c36 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:54.894490 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebcf688-92b7-47be-9263-6268c4b775f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:54.895551 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebcf688-92b7-47be-9263-6268c4b775f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:54.895551 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebcf688-92b7-47be-9263-6268c4b775f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:54.895715 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebcf688-92b7-47be-9263-6268c4b775f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:54.925316 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebcf688-92b7-47be-9263-6268c4b775f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:54.927521 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebcf688-92b7-47be-9263-6268c4b775f7 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:54.943619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1324/2654] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:58:56.881298 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b310ad16-b58d-4acc-9479-444cf56437c6 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:56.924387 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:56.951465 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b310ad16-b58d-4acc-9479-444cf56437c6 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:56.955958 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccaa979-1b7c-4a1b-87bb-eb3a04c47881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:56.956583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccaa979-1b7c-4a1b-87bb-eb3a04c47881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:56.956860 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccaa979-1b7c-4a1b-87bb-eb3a04c47881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:56.957132 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccaa979-1b7c-4a1b-87bb-eb3a04c47881 admin admin] PATH_INFO: `/v3/users/46728ca0b6c347078c0e6d1862ff94ce` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:56.958249 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccaa979-1b7c-4a1b-87bb-eb3a04c47881 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46728ca0b6c347078c0e6d1862ff94ce)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:56.960242 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccaa979-1b7c-4a1b-87bb-eb3a04c47881 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:57.012293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-cccaa979-1b7c-4a1b-87bb-eb3a04c47881 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46728ca0b6c347078c0e6d1862ff94ce'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:57.020548 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-cccaa979-1b7c-4a1b-87bb-eb3a04c47881 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46728ca0b6c347078c0e6d1862ff94ce'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:57.028081 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-cccaa979-1b7c-4a1b-87bb-eb3a04c47881 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46728ca0b6c347078c0e6d1862ff94ce'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:57.028747 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-cccaa979-1b7c-4a1b-87bb-eb3a04c47881 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46728ca0b6c347078c0e6d1862ff94ce'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:57.029401 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-cccaa979-1b7c-4a1b-87bb-eb3a04c47881 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46728ca0b6c347078c0e6d1862ff94ce'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:57.047419 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1331/2655] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:58:56 2022] DELETE /identity/v3/users/46728ca0b6c347078c0e6d1862ff94ce => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:57.051864 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3ed95-35ce-46c8-b6c1-9680acaade0c None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:57.052307 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3ed95-35ce-46c8-b6c1-9680acaade0c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:57.052388 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3ed95-35ce-46c8-b6c1-9680acaade0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:57.071604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-98d3ed95-35ce-46c8-b6c1-9680acaade0c None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:57.072771 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-98d3ed95-35ce-46c8-b6c1-9680acaade0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:57.101449 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:57.146157 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1325/2656] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:57.156881 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccaa979-1b7c-4a1b-87bb-eb3a04c47881 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:57.158315 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-cccaa979-1b7c-4a1b-87bb-eb3a04c47881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:57.227670 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccaa979-1b7c-4a1b-87bb-eb3a04c47881 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:57.236610 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c4b6f-6fba-4537-9f78-59741d77a1dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:57.237247 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c4b6f-6fba-4537-9f78-59741d77a1dc service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:57.237532 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c4b6f-6fba-4537-9f78-59741d77a1dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:57.237808 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c4b6f-6fba-4537-9f78-59741d77a1dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:57.249347 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c4b6f-6fba-4537-9f78-59741d77a1dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:57.250851 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c4b6f-6fba-4537-9f78-59741d77a1dc service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:57.261804 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1332/2657] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:58:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:58:57.605110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d3ed95-35ce-46c8-b6c1-9680acaade0c None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:57.607429 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-98d3ed95-35ce-46c8-b6c1-9680acaade0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:57.622570 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d3ed95-35ce-46c8-b6c1-9680acaade0c None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:57.627436 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7ba1c-b5f1-4f16-883d-037ba177d26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:57.628465 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7ba1c-b5f1-4f16-883d-037ba177d26c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:57.628746 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7ba1c-b5f1-4f16-883d-037ba177d26c admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:57.629089 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7ba1c-b5f1-4f16-883d-037ba177d26c admin admin] PATH_INFO: `/v3/projects/bd0c7b2d6c6e4ce487bc08422bcde3e9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:57.630161 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7ba1c-b5f1-4f16-883d-037ba177d26c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd0c7b2d6c6e4ce487bc08422bcde3e9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:57.632659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7ba1c-b5f1-4f16-883d-037ba177d26c admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:57.678249 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-cbb7ba1c-b5f1-4f16-883d-037ba177d26c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd0c7b2d6c6e4ce487bc08422bcde3e9', 'request_id': 'req-cbb7ba1c-b5f1-4f16-883d-037ba177d26c'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:57.684765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-cbb7ba1c-b5f1-4f16-883d-037ba177d26c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd0c7b2d6c6e4ce487bc08422bcde3e9', 'request_id': 'req-cbb7ba1c-b5f1-4f16-883d-037ba177d26c'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:57.690209 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-cbb7ba1c-b5f1-4f16-883d-037ba177d26c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd0c7b2d6c6e4ce487bc08422bcde3e9', 'request_id': 'req-cbb7ba1c-b5f1-4f16-883d-037ba177d26c'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:57.701690 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-cbb7ba1c-b5f1-4f16-883d-037ba177d26c admin admin] The token cache is being invalidate because project bd0c7b2d6c6e4ce487bc08422bcde3e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:58:57.701917 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-cbb7ba1c-b5f1-4f16-883d-037ba177d26c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:57.702566 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-cbb7ba1c-b5f1-4f16-883d-037ba177d26c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:57.704074 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1326/2658] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:58:57 2022] DELETE /identity/v3/projects/bd0c7b2d6c6e4ce487bc08422bcde3e9 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:57.708036 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c4b6f-6fba-4537-9f78-59741d77a1dc service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:57.720860 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c4b6f-6fba-4537-9f78-59741d77a1dc service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:57.775057 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:57.782331 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf99015-ca6d-4441-9895-0c952871d99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:57.783169 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf99015-ca6d-4441-9895-0c952871d99f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:57.783618 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf99015-ca6d-4441-9895-0c952871d99f admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:57.784065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf99015-ca6d-4441-9895-0c952871d99f admin admin] PATH_INFO: `/v3/users/75fad4bb2b5a44779e527d42617a02d3` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:57.785402 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf99015-ca6d-4441-9895-0c952871d99f admin admin] RBAC: Authorizing `identity:delete_user(user_id=75fad4bb2b5a44779e527d42617a02d3)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:57.787371 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf99015-ca6d-4441-9895-0c952871d99f admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:57.835810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-1cf99015-ca6d-4441-9895-0c952871d99f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75fad4bb2b5a44779e527d42617a02d3'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:57.847231 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-1cf99015-ca6d-4441-9895-0c952871d99f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75fad4bb2b5a44779e527d42617a02d3'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:57.856330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-1cf99015-ca6d-4441-9895-0c952871d99f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75fad4bb2b5a44779e527d42617a02d3'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:57.857054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-1cf99015-ca6d-4441-9895-0c952871d99f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75fad4bb2b5a44779e527d42617a02d3'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:57.857666 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-1cf99015-ca6d-4441-9895-0c952871d99f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75fad4bb2b5a44779e527d42617a02d3'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:57.875226 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1333/2659] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:58:57 2022] DELETE /identity/v3/users/75fad4bb2b5a44779e527d42617a02d3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:58.233982 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7ba1c-b5f1-4f16-883d-037ba177d26c admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:58.235472 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-cbb7ba1c-b5f1-4f16-883d-037ba177d26c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:58.255217 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7ba1c-b5f1-4f16-883d-037ba177d26c admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:58.313479 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de509fa0-6e28-4b49-8aa4-a3d331b88c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:58.314135 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de509fa0-6e28-4b49-8aa4-a3d331b88c13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:58.314570 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de509fa0-6e28-4b49-8aa4-a3d331b88c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:58.314757 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de509fa0-6e28-4b49-8aa4-a3d331b88c13 admin admin] PATH_INFO: `/v3/projects/489f153f2f154c3b9811b08d2e3cbb80` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:58.316027 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de509fa0-6e28-4b49-8aa4-a3d331b88c13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=489f153f2f154c3b9811b08d2e3cbb80)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:58.317935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de509fa0-6e28-4b49-8aa4-a3d331b88c13 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:58.367237 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-de509fa0-6e28-4b49-8aa4-a3d331b88c13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '489f153f2f154c3b9811b08d2e3cbb80', 'request_id': 'req-de509fa0-6e28-4b49-8aa4-a3d331b88c13'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:58.375377 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-de509fa0-6e28-4b49-8aa4-a3d331b88c13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '489f153f2f154c3b9811b08d2e3cbb80', 'request_id': 'req-de509fa0-6e28-4b49-8aa4-a3d331b88c13'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:58.383300 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:58.385235 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-de509fa0-6e28-4b49-8aa4-a3d331b88c13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '489f153f2f154c3b9811b08d2e3cbb80', 'request_id': 'req-de509fa0-6e28-4b49-8aa4-a3d331b88c13'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:58.400937 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-de509fa0-6e28-4b49-8aa4-a3d331b88c13 admin admin] The token cache is being invalidate because project 489f153f2f154c3b9811b08d2e3cbb80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:58:58.401235 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-de509fa0-6e28-4b49-8aa4-a3d331b88c13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:58.401891 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-de509fa0-6e28-4b49-8aa4-a3d331b88c13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:58:58.403414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1327/2660] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:58:58 2022] DELETE /identity/v3/projects/489f153f2f154c3b9811b08d2e3cbb80 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:58.408412 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d292ce3-ada4-4be0-964c-be742edab839 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:58.408889 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d292ce3-ada4-4be0-964c-be742edab839 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:58.409091 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d292ce3-ada4-4be0-964c-be742edab839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:58.431578 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-0d292ce3-ada4-4be0-964c-be742edab839 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:58.432781 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-0d292ce3-ada4-4be0-964c-be742edab839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:58.495976 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1334/2661] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:58.503747 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2caf156-eb5d-43a9-8b36-ee22c1c6b71f None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:58.504039 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2caf156-eb5d-43a9-8b36-ee22c1c6b71f None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:58.504313 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2caf156-eb5d-43a9-8b36-ee22c1c6b71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:58.524752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-c2caf156-eb5d-43a9-8b36-ee22c1c6b71f None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:58.525981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c2caf156-eb5d-43a9-8b36-ee22c1c6b71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:58.534743 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1328/2662] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:58.539886 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d292ce3-ada4-4be0-964c-be742edab839 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:58.550086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d292ce3-ada4-4be0-964c-be742edab839 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:58.559053 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2008975-3ee5-4f66-a70e-a62197b0709c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:58.559677 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2008975-3ee5-4f66-a70e-a62197b0709c admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:58.559958 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2008975-3ee5-4f66-a70e-a62197b0709c admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:58.560228 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2008975-3ee5-4f66-a70e-a62197b0709c admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:58.560871 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2008975-3ee5-4f66-a70e-a62197b0709c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:58:58.561174 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2008975-3ee5-4f66-a70e-a62197b0709c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:58.562773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2008975-3ee5-4f66-a70e-a62197b0709c admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:58.574602 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1335/2663] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 00:58:58.578952 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2caf156-eb5d-43a9-8b36-ee22c1c6b71f None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:58.588251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2caf156-eb5d-43a9-8b36-ee22c1c6b71f None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:58.593291 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588dd78f-c7dc-4bb3-bac2-9730a32b7c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:58.593887 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588dd78f-c7dc-4bb3-bac2-9730a32b7c1e admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:58.594432 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588dd78f-c7dc-4bb3-bac2-9730a32b7c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:58.594730 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588dd78f-c7dc-4bb3-bac2-9730a32b7c1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:58.595382 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588dd78f-c7dc-4bb3-bac2-9730a32b7c1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:58.597260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588dd78f-c7dc-4bb3-bac2-9730a32b7c1e admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:58.628445 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1329/2664] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:58:58 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:58:58.632872 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2008975-3ee5-4f66-a70e-a62197b0709c admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:58.642881 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2008975-3ee5-4f66-a70e-a62197b0709c admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:58.699198 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e12d9-f356-42b9-b8ab-5d0414c11c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:58.699806 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e12d9-f356-42b9-b8ab-5d0414c11c61 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:58.700085 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e12d9-f356-42b9-b8ab-5d0414c11c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:58.700358 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e12d9-f356-42b9-b8ab-5d0414c11c61 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:58.700980 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e12d9-f356-42b9-b8ab-5d0414c11c61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:58.702898 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e12d9-f356-42b9-b8ab-5d0414c11c61 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:58.713381 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:58.725245 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1336/2665] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:58:58 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:58:58.729956 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588dd78f-c7dc-4bb3-bac2-9730a32b7c1e admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:58.740135 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588dd78f-c7dc-4bb3-bac2-9730a32b7c1e admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:58.744922 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f4037-b766-4e27-91dc-0b6a4aa9ee6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:58.745548 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f4037-b766-4e27-91dc-0b6a4aa9ee6d admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:58.745823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f4037-b766-4e27-91dc-0b6a4aa9ee6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:58.746091 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f4037-b766-4e27-91dc-0b6a4aa9ee6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:58.746719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f4037-b766-4e27-91dc-0b6a4aa9ee6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:58.748341 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f4037-b766-4e27-91dc-0b6a4aa9ee6d admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:58.758453 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1330/2666] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:58:58.762470 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e12d9-f356-42b9-b8ab-5d0414c11c61 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:58.772961 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e12d9-f356-42b9-b8ab-5d0414c11c61 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:58.778098 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626dd77-4e62-4a18-9fc2-c5c992331e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:58.778729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626dd77-4e62-4a18-9fc2-c5c992331e7b admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:58.779009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626dd77-4e62-4a18-9fc2-c5c992331e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:58.779311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626dd77-4e62-4a18-9fc2-c5c992331e7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:58.779951 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d626dd77-4e62-4a18-9fc2-c5c992331e7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:58.781547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d626dd77-4e62-4a18-9fc2-c5c992331e7b admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:58.793148 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1337/2667] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:58:58.797368 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f4037-b766-4e27-91dc-0b6a4aa9ee6d admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:58.808546 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f4037-b766-4e27-91dc-0b6a4aa9ee6d admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:58.813580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e574a4-a7fe-461c-94e9-cd5bddb8fde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:58.814250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e574a4-a7fe-461c-94e9-cd5bddb8fde1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:58.814531 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e574a4-a7fe-461c-94e9-cd5bddb8fde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:58.814808 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e574a4-a7fe-461c-94e9-cd5bddb8fde1 admin admin] PATH_INFO: `/v3/projects/877f6640abb54661b85b55e537a3ab3d/users/0ef3e5deb196495a8f18f2048e13ee88/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:58.830454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e574a4-a7fe-461c-94e9-cd5bddb8fde1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=877f6640abb54661b85b55e537a3ab3d, user_id=0ef3e5deb196495a8f18f2048e13ee88, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:58.833140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e574a4-a7fe-461c-94e9-cd5bddb8fde1 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:58.847349 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1331/2668] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:58:58 2022] PUT /identity/v3/projects/877f6640abb54661b85b55e537a3ab3d/users/0ef3e5deb196495a8f18f2048e13ee88/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:58.861575 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229623d-b8c4-4449-8d2a-ee64ecb940d4 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:58.861857 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229623d-b8c4-4449-8d2a-ee64ecb940d4 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:58.862126 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229623d-b8c4-4449-8d2a-ee64ecb940d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:58.892033 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-3229623d-b8c4-4449-8d2a-ee64ecb940d4 None None] MFA Rules not processed for user `0ef3e5deb196495a8f18f2048e13ee88`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:58.893285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-3229623d-b8c4-4449-8d2a-ee64ecb940d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:58.962086 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1338/2669] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:58.966671 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e574a4-a7fe-461c-94e9-cd5bddb8fde1 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:59.034857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e574a4-a7fe-461c-94e9-cd5bddb8fde1 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:59.040339 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a4edc-6519-42a3-884f-a45ab14b0996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:59.041023 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a4edc-6519-42a3-884f-a45ab14b0996 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:59.041304 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a4edc-6519-42a3-884f-a45ab14b0996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:59.041570 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a4edc-6519-42a3-884f-a45ab14b0996 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:59.042202 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a4edc-6519-42a3-884f-a45ab14b0996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:59.044115 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a4edc-6519-42a3-884f-a45ab14b0996 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:59.073475 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1332/2670] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:58:58 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:58:59.077602 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229623d-b8c4-4449-8d2a-ee64ecb940d4 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:59.089293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229623d-b8c4-4449-8d2a-ee64ecb940d4 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:59.190263 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec850f-5a9c-44cb-8811-bd2c4d7f52ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:59.191010 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec850f-5a9c-44cb-8811-bd2c4d7f52ed admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:59.191341 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec850f-5a9c-44cb-8811-bd2c4d7f52ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:59.191617 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec850f-5a9c-44cb-8811-bd2c4d7f52ed admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:59.192330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec850f-5a9c-44cb-8811-bd2c4d7f52ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:59.194658 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec850f-5a9c-44cb-8811-bd2c4d7f52ed admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:59.223622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a4edc-6519-42a3-884f-a45ab14b0996 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:59.224359 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1339/2671] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:58:59 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:58:59.235265 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a4edc-6519-42a3-884f-a45ab14b0996 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:59.240240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4535e02-35ab-43c4-9031-3e8c6ee0f635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:59.240866 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4535e02-35ab-43c4-9031-3e8c6ee0f635 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:59.241816 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4535e02-35ab-43c4-9031-3e8c6ee0f635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:59.242722 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4535e02-35ab-43c4-9031-3e8c6ee0f635 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:59.242722 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4535e02-35ab-43c4-9031-3e8c6ee0f635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:59.244444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4535e02-35ab-43c4-9031-3e8c6ee0f635 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:59.268755 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1333/2672] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:58:59.273956 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec850f-5a9c-44cb-8811-bd2c4d7f52ed admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:59.284535 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec850f-5a9c-44cb-8811-bd2c4d7f52ed admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:59.292580 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d219923d-7ea0-4310-9c12-0e32b26f1989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:59.293240 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219923d-7ea0-4310-9c12-0e32b26f1989 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:59.293596 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219923d-7ea0-4310-9c12-0e32b26f1989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:59.293970 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219923d-7ea0-4310-9c12-0e32b26f1989 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:59.294703 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d219923d-7ea0-4310-9c12-0e32b26f1989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:59.296447 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d219923d-7ea0-4310-9c12-0e32b26f1989 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:59.307791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1340/2673] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:58:59.311880 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4535e02-35ab-43c4-9031-3e8c6ee0f635 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:59.323172 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4535e02-35ab-43c4-9031-3e8c6ee0f635 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:59.328265 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6a8c5-7721-4d6c-ad6e-b4df88d3858c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:59.329184 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6a8c5-7721-4d6c-ad6e-b4df88d3858c admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:59.329900 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6a8c5-7721-4d6c-ad6e-b4df88d3858c admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:59.330246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6a8c5-7721-4d6c-ad6e-b4df88d3858c admin admin] PATH_INFO: `/v3/projects/ad2bb8db385f4f43801291644627b67f/users/1004d160f2cc42aa84431dc125720593/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:59.351470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6a8c5-7721-4d6c-ad6e-b4df88d3858c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad2bb8db385f4f43801291644627b67f, user_id=1004d160f2cc42aa84431dc125720593, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:59.354221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6a8c5-7721-4d6c-ad6e-b4df88d3858c admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:59.369069 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1334/2674] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:58:59 2022] PUT /identity/v3/projects/ad2bb8db385f4f43801291644627b67f/users/1004d160f2cc42aa84431dc125720593/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:58:59.381788 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247473a-90be-4316-bf79-30cbac883c35 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:59.382154 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247473a-90be-4316-bf79-30cbac883c35 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:59.382559 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247473a-90be-4316-bf79-30cbac883c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:59.420676 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-0247473a-90be-4316-bf79-30cbac883c35 None None] MFA Rules not processed for user `1004d160f2cc42aa84431dc125720593`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:58:59.422141 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-0247473a-90be-4316-bf79-30cbac883c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:58:59.469387 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:59.510965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1341/2675] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:58:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:58:59.515738 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6a8c5-7721-4d6c-ad6e-b4df88d3858c admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:59.563604 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:58:59.598787 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6a8c5-7721-4d6c-ad6e-b4df88d3858c admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:59.605639 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac56a2-85cf-4ab4-86dc-2dfa5da8e93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:59.606546 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac56a2-85cf-4ab4-86dc-2dfa5da8e93c admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:59.606922 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac56a2-85cf-4ab4-86dc-2dfa5da8e93c admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:59.607400 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac56a2-85cf-4ab4-86dc-2dfa5da8e93c admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:59.608392 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac56a2-85cf-4ab4-86dc-2dfa5da8e93c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:59.610445 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac56a2-85cf-4ab4-86dc-2dfa5da8e93c admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:59.659961 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1335/2676] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:58:59 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:58:59.664271 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247473a-90be-4316-bf79-30cbac883c35 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:59.675468 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247473a-90be-4316-bf79-30cbac883c35 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:59.766764 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a088c-ad7d-4682-9cce-d63b5999895a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:59.767862 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a088c-ad7d-4682-9cce-d63b5999895a admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:59.768150 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a088c-ad7d-4682-9cce-d63b5999895a admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:59.768420 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a088c-ad7d-4682-9cce-d63b5999895a admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:59.769065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a088c-ad7d-4682-9cce-d63b5999895a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:59.771016 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a088c-ad7d-4682-9cce-d63b5999895a admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:59.798220 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1342/2677] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:58:59 2022] POST /identity/v3/users => generated 298 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:58:59.804443 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac56a2-85cf-4ab4-86dc-2dfa5da8e93c admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:59.814135 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac56a2-85cf-4ab4-86dc-2dfa5da8e93c admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:59.830936 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799d014-bbe3-40ae-a111-6713c2ebc8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:59.832217 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799d014-bbe3-40ae-a111-6713c2ebc8a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:59.832217 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799d014-bbe3-40ae-a111-6713c2ebc8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:59.832217 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799d014-bbe3-40ae-a111-6713c2ebc8a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:59.832922 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d799d014-bbe3-40ae-a111-6713c2ebc8a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:59.834939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d799d014-bbe3-40ae-a111-6713c2ebc8a1 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:59.847595 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1336/2678] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:58:59.853254 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a088c-ad7d-4682-9cce-d63b5999895a admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:59.869046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a088c-ad7d-4682-9cce-d63b5999895a admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:58:59.883260 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9c048-337f-4225-89e6-adf9f59fce7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:58:59.884148 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9c048-337f-4225-89e6-adf9f59fce7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:58:59.884537 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9c048-337f-4225-89e6-adf9f59fce7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:58:59.884920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9c048-337f-4225-89e6-adf9f59fce7c admin admin] PATH_INFO: `/v3/projects/0f4ab2958efc4806beee8fb71d2a0823/users/3234a23b52bf48c1862a86fa933297c5/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:58:59.906047 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec9c048-337f-4225-89e6-adf9f59fce7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f4ab2958efc4806beee8fb71d2a0823, user_id=3234a23b52bf48c1862a86fa933297c5, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:58:59.909057 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec9c048-337f-4225-89e6-adf9f59fce7c admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:58:59.931779 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1343/2679] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:58:59 2022] PUT /identity/v3/projects/0f4ab2958efc4806beee8fb71d2a0823/users/3234a23b52bf48c1862a86fa933297c5/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:58:59.935388 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799d014-bbe3-40ae-a111-6713c2ebc8a1 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:58:59.946513 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799d014-bbe3-40ae-a111-6713c2ebc8a1 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:00.091421 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594eff95-983d-4303-a3f2-8346d7bd9b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:00.092737 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594eff95-983d-4303-a3f2-8346d7bd9b07 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:00.093127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594eff95-983d-4303-a3f2-8346d7bd9b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:00.093464 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594eff95-983d-4303-a3f2-8346d7bd9b07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:00.094517 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594eff95-983d-4303-a3f2-8346d7bd9b07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:00.105652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594eff95-983d-4303-a3f2-8346d7bd9b07 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:00.118034 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1337/2680] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:59:00.122115 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9c048-337f-4225-89e6-adf9f59fce7c admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:00.140488 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9c048-337f-4225-89e6-adf9f59fce7c admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:00.159269 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a8034-446e-4608-9cf3-cf1658ba8134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:00.160076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a8034-446e-4608-9cf3-cf1658ba8134 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:00.161216 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a8034-446e-4608-9cf3-cf1658ba8134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:00.161978 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a8034-446e-4608-9cf3-cf1658ba8134 admin admin] PATH_INFO: `/v3/projects/0f4ab2958efc4806beee8fb71d2a0823/users/3234a23b52bf48c1862a86fa933297c5/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:00.166224 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45a8034-446e-4608-9cf3-cf1658ba8134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f4ab2958efc4806beee8fb71d2a0823, user_id=3234a23b52bf48c1862a86fa933297c5, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:00.170941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45a8034-446e-4608-9cf3-cf1658ba8134 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:00.187062 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1344/2681] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:59:00 2022] PUT /identity/v3/projects/0f4ab2958efc4806beee8fb71d2a0823/users/3234a23b52bf48c1862a86fa933297c5/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:59:00.198651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445cc09-6301-4109-8f54-a9fcad928494 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:00.199088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445cc09-6301-4109-8f54-a9fcad928494 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:00.199347 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445cc09-6301-4109-8f54-a9fcad928494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:00.234855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-4445cc09-6301-4109-8f54-a9fcad928494 None None] MFA Rules not processed for user `3234a23b52bf48c1862a86fa933297c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:59:00.236542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-4445cc09-6301-4109-8f54-a9fcad928494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:00.323071 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1338/2682] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:59:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:59:00.334150 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a8034-446e-4608-9cf3-cf1658ba8134 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:00.336281 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f45a8034-446e-4608-9cf3-cf1658ba8134 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:00.405982 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a8034-446e-4608-9cf3-cf1658ba8134 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:00.409742 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb92df8-79a7-4e6d-9b03-0957c347ef9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:00.410340 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb92df8-79a7-4e6d-9b03-0957c347ef9b service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:00.410623 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb92df8-79a7-4e6d-9b03-0957c347ef9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:00.411713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb92df8-79a7-4e6d-9b03-0957c347ef9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:00.465569 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:59:00.486292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb92df8-79a7-4e6d-9b03-0957c347ef9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:00.488026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb92df8-79a7-4e6d-9b03-0957c347ef9b service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:00.498982 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1345/2683] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:59:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:59:00.876547 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0ad0d-3f97-49e6-bfb5-f4eb59117911 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:00.877017 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0ad0d-3f97-49e6-bfb5-f4eb59117911 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:00.877433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0ad0d-3f97-49e6-bfb5-f4eb59117911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:00.903646 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-22e0ad0d-3f97-49e6-bfb5-f4eb59117911 None None] MFA Rules not processed for user `a8fb7a9acefd4b418c992661037c2ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:59:00.904870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-22e0ad0d-3f97-49e6-bfb5-f4eb59117911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:00.941378 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:59:00.989995 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1339/2684] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 31 00:59:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:59:01.003013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb92df8-79a7-4e6d-9b03-0957c347ef9b service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:01.004511 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4bb92df8-79a7-4e6d-9b03-0957c347ef9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:01.090846 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb92df8-79a7-4e6d-9b03-0957c347ef9b service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:01.098688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee243c2-a41b-4a2b-933a-ef1b13b0960d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd3cfbc8ebd4c5a83bef09efe10bc91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:01.099448 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee243c2-a41b-4a2b-933a-ef1b13b0960d service swift] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:01.099805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee243c2-a41b-4a2b-933a-ef1b13b0960d service swift] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:01.100151 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee243c2-a41b-4a2b-933a-ef1b13b0960d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:01.110813 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee243c2-a41b-4a2b-933a-ef1b13b0960d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:01.112469 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee243c2-a41b-4a2b-933a-ef1b13b0960d service swift] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:01.125272 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1346/2685] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 31 00:59:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:59:02.023798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0ad0d-3f97-49e6-bfb5-f4eb59117911 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:02.025456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-22e0ad0d-3f97-49e6-bfb5-f4eb59117911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:02.109820 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0ad0d-3f97-49e6-bfb5-f4eb59117911 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:02.114519 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e267163-550a-411b-9531-edcfb662e6cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:02.115433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e267163-550a-411b-9531-edcfb662e6cc service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:02.116110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e267163-550a-411b-9531-edcfb662e6cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:02.117245 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e267163-550a-411b-9531-edcfb662e6cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:02.119269 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-2e267163-550a-411b-9531-edcfb662e6cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:02.227712 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee243c2-a41b-4a2b-933a-ef1b13b0960d service swift] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:02.229548 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-dee243c2-a41b-4a2b-933a-ef1b13b0960d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:02.244804 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee243c2-a41b-4a2b-933a-ef1b13b0960d service swift] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:02.260309 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177de7b-08a8-46d4-9297-ad5077b19d97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:02.260944 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177de7b-08a8-46d4-9297-ad5077b19d97 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:02.261229 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177de7b-08a8-46d4-9297-ad5077b19d97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:02.261500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177de7b-08a8-46d4-9297-ad5077b19d97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:02.276226 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e267163-550a-411b-9531-edcfb662e6cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:02.280524 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e267163-550a-411b-9531-edcfb662e6cc service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:02.293777 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1340/2686] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:59:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:59:02.406893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177de7b-08a8-46d4-9297-ad5077b19d97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:02.408752 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177de7b-08a8-46d4-9297-ad5077b19d97 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:02.434754 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1347/2687] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:59:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:59:03.272903 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e267163-550a-411b-9531-edcfb662e6cc service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:03.283151 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e267163-550a-411b-9531-edcfb662e6cc service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:03.287021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4bcf8-73be-4b9f-bdd4-0405147cfe2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:03.287849 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4bcf8-73be-4b9f-bdd4-0405147cfe2c service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:03.288434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4bcf8-73be-4b9f-bdd4-0405147cfe2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:03.288862 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4bcf8-73be-4b9f-bdd4-0405147cfe2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:03.299557 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d4bcf8-73be-4b9f-bdd4-0405147cfe2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:03.301093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d4bcf8-73be-4b9f-bdd4-0405147cfe2c service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:03.312495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1341/2688] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:59:03 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:59:04.939981 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177de7b-08a8-46d4-9297-ad5077b19d97 service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:04.961501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177de7b-08a8-46d4-9297-ad5077b19d97 service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:04.965531 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b7dda-c319-4d65-8f18-f9b3ff057014 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:04.966248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b7dda-c319-4d65-8f18-f9b3ff057014 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:04.966633 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b7dda-c319-4d65-8f18-f9b3ff057014 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:04.966993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b7dda-c319-4d65-8f18-f9b3ff057014 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:04.970724 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a04b7dda-c319-4d65-8f18-f9b3ff057014 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:05.008346 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b7dda-c319-4d65-8f18-f9b3ff057014 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:05.009899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b7dda-c319-4d65-8f18-f9b3ff057014 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:05.028842 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1348/2689] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:59:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:59:08.443067 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4bcf8-73be-4b9f-bdd4-0405147cfe2c service cinder] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:08.550426 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d4bcf8-73be-4b9f-bdd4-0405147cfe2c service cinder] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:08.559020 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44bf98-a6aa-4a7b-acd4-b584fa52671e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:08.559719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44bf98-a6aa-4a7b-acd4-b584fa52671e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:08.560193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44bf98-a6aa-4a7b-acd4-b584fa52671e admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:08.560501 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44bf98-a6aa-4a7b-acd4-b584fa52671e admin admin] PATH_INFO: `/v3/users/0ef3e5deb196495a8f18f2048e13ee88` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:08.561718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b44bf98-a6aa-4a7b-acd4-b584fa52671e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ef3e5deb196495a8f18f2048e13ee88)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:08.563725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b44bf98-a6aa-4a7b-acd4-b584fa52671e admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:08.566482 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:59:08.620895 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-4b44bf98-a6aa-4a7b-acd4-b584fa52671e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ef3e5deb196495a8f18f2048e13ee88'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:08.637117 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-4b44bf98-a6aa-4a7b-acd4-b584fa52671e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ef3e5deb196495a8f18f2048e13ee88'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:08.644552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-4b44bf98-a6aa-4a7b-acd4-b584fa52671e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ef3e5deb196495a8f18f2048e13ee88'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:08.653132 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-4b44bf98-a6aa-4a7b-acd4-b584fa52671e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ef3e5deb196495a8f18f2048e13ee88'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:08.653842 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-4b44bf98-a6aa-4a7b-acd4-b584fa52671e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ef3e5deb196495a8f18f2048e13ee88'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:08.660251 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1342/2690] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:59:08 2022] DELETE /identity/v3/users/0ef3e5deb196495a8f18f2048e13ee88 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:59:08.665285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accfdd7a-fde5-4a50-8bf5-e433257b06dd None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:08.665816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accfdd7a-fde5-4a50-8bf5-e433257b06dd None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:08.665816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accfdd7a-fde5-4a50-8bf5-e433257b06dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:08.686230 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-accfdd7a-fde5-4a50-8bf5-e433257b06dd None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:59:08.687622 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-accfdd7a-fde5-4a50-8bf5-e433257b06dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:08.752653 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1349/2691] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:59:08.763125 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44bf98-a6aa-4a7b-acd4-b584fa52671e admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:08.764541 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-4b44bf98-a6aa-4a7b-acd4-b584fa52671e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:08.826038 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44bf98-a6aa-4a7b-acd4-b584fa52671e admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:08.829844 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2633d-c452-4e41-b3f4-5265635b90e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:08.830463 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2633d-c452-4e41-b3f4-5265635b90e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:08.830872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2633d-c452-4e41-b3f4-5265635b90e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:08.831190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2633d-c452-4e41-b3f4-5265635b90e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:08.841834 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2633d-c452-4e41-b3f4-5265635b90e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:08.843316 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2633d-c452-4e41-b3f4-5265635b90e2 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:08.853941 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1343/2692] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:59:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 00:59:09.136486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accfdd7a-fde5-4a50-8bf5-e433257b06dd None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:09.138110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-accfdd7a-fde5-4a50-8bf5-e433257b06dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:09.146011 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:59:09.154459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accfdd7a-fde5-4a50-8bf5-e433257b06dd None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:09.160927 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de4d90-3262-4b47-8402-9ea5c13b887c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:09.161151 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de4d90-3262-4b47-8402-9ea5c13b887c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:09.161301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de4d90-3262-4b47-8402-9ea5c13b887c admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:09.161663 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de4d90-3262-4b47-8402-9ea5c13b887c admin admin] PATH_INFO: `/v3/projects/877f6640abb54661b85b55e537a3ab3d` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:09.163497 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de4d90-3262-4b47-8402-9ea5c13b887c admin admin] RBAC: Authorizing `identity:delete_project(project_id=877f6640abb54661b85b55e537a3ab3d)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:09.165542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de4d90-3262-4b47-8402-9ea5c13b887c admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:09.219029 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-a5de4d90-3262-4b47-8402-9ea5c13b887c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '877f6640abb54661b85b55e537a3ab3d', 'request_id': 'req-a5de4d90-3262-4b47-8402-9ea5c13b887c'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:09.226012 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-a5de4d90-3262-4b47-8402-9ea5c13b887c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '877f6640abb54661b85b55e537a3ab3d', 'request_id': 'req-a5de4d90-3262-4b47-8402-9ea5c13b887c'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:09.232885 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-a5de4d90-3262-4b47-8402-9ea5c13b887c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '877f6640abb54661b85b55e537a3ab3d', 'request_id': 'req-a5de4d90-3262-4b47-8402-9ea5c13b887c'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:09.254855 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-a5de4d90-3262-4b47-8402-9ea5c13b887c admin admin] The token cache is being invalidate because project 877f6640abb54661b85b55e537a3ab3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:59:09.255263 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-a5de4d90-3262-4b47-8402-9ea5c13b887c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:09.255949 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-a5de4d90-3262-4b47-8402-9ea5c13b887c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:09.258289 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1350/2693] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:59:09 2022] DELETE /identity/v3/projects/877f6640abb54661b85b55e537a3ab3d => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:59:09.262116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2633d-c452-4e41-b3f4-5265635b90e2 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:09.271827 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2633d-c452-4e41-b3f4-5265635b90e2 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:09.337138 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb70d9-7a0f-484a-a77a-e6fe4d4695cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:09.337884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb70d9-7a0f-484a-a77a-e6fe4d4695cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:09.338213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb70d9-7a0f-484a-a77a-e6fe4d4695cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:09.338551 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb70d9-7a0f-484a-a77a-e6fe4d4695cf admin admin] PATH_INFO: `/v3/users/1004d160f2cc42aa84431dc125720593` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:09.339772 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb70d9-7a0f-484a-a77a-e6fe4d4695cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=1004d160f2cc42aa84431dc125720593)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:09.341812 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb70d9-7a0f-484a-a77a-e6fe4d4695cf admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:09.370441 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:59:09.389653 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-59eb70d9-7a0f-484a-a77a-e6fe4d4695cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1004d160f2cc42aa84431dc125720593'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:09.401455 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-59eb70d9-7a0f-484a-a77a-e6fe4d4695cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1004d160f2cc42aa84431dc125720593'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:09.408161 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-59eb70d9-7a0f-484a-a77a-e6fe4d4695cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1004d160f2cc42aa84431dc125720593'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:09.415428 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-59eb70d9-7a0f-484a-a77a-e6fe4d4695cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1004d160f2cc42aa84431dc125720593'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:09.416035 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-59eb70d9-7a0f-484a-a77a-e6fe4d4695cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1004d160f2cc42aa84431dc125720593'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:09.421934 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1344/2694] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:59:09 2022] DELETE /identity/v3/users/1004d160f2cc42aa84431dc125720593 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:59:09.699891 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de4d90-3262-4b47-8402-9ea5c13b887c admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:09.701421 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a5de4d90-3262-4b47-8402-9ea5c13b887c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:09.715422 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de4d90-3262-4b47-8402-9ea5c13b887c admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:09.774100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e0f15-ddab-4541-8017-f5d43581a1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:09.774789 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e0f15-ddab-4541-8017-f5d43581a1e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:09.775095 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e0f15-ddab-4541-8017-f5d43581a1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:09.775381 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e0f15-ddab-4541-8017-f5d43581a1e4 admin admin] PATH_INFO: `/v3/projects/ad2bb8db385f4f43801291644627b67f` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:09.776530 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e0f15-ddab-4541-8017-f5d43581a1e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad2bb8db385f4f43801291644627b67f)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:09.778503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e0f15-ddab-4541-8017-f5d43581a1e4 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:09.828888 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8d3e0f15-ddab-4541-8017-f5d43581a1e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad2bb8db385f4f43801291644627b67f', 'request_id': 'req-8d3e0f15-ddab-4541-8017-f5d43581a1e4'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:09.834551 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8d3e0f15-ddab-4541-8017-f5d43581a1e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad2bb8db385f4f43801291644627b67f', 'request_id': 'req-8d3e0f15-ddab-4541-8017-f5d43581a1e4'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:09.841378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8d3e0f15-ddab-4541-8017-f5d43581a1e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad2bb8db385f4f43801291644627b67f', 'request_id': 'req-8d3e0f15-ddab-4541-8017-f5d43581a1e4'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:09.854793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8d3e0f15-ddab-4541-8017-f5d43581a1e4 admin admin] The token cache is being invalidate because project ad2bb8db385f4f43801291644627b67f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:59:09.855000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8d3e0f15-ddab-4541-8017-f5d43581a1e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:09.855644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-8d3e0f15-ddab-4541-8017-f5d43581a1e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:09.857110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1351/2695] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:59:09 2022] DELETE /identity/v3/projects/ad2bb8db385f4f43801291644627b67f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:59:09.861172 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb70d9-7a0f-484a-a77a-e6fe4d4695cf admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:09.862659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-59eb70d9-7a0f-484a-a77a-e6fe4d4695cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:09.876359 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb70d9-7a0f-484a-a77a-e6fe4d4695cf admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:09.933238 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe548dd-2519-4a00-afe0-e1882e4977de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:09.933857 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe548dd-2519-4a00-afe0-e1882e4977de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:09.934131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe548dd-2519-4a00-afe0-e1882e4977de admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:09.934402 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe548dd-2519-4a00-afe0-e1882e4977de admin admin] PATH_INFO: `/v3/users/3234a23b52bf48c1862a86fa933297c5` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:09.935526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe548dd-2519-4a00-afe0-e1882e4977de admin admin] RBAC: Authorizing `identity:delete_user(user_id=3234a23b52bf48c1862a86fa933297c5)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:09.937413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe548dd-2519-4a00-afe0-e1882e4977de admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:09.983470 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ebe548dd-2519-4a00-afe0-e1882e4977de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3234a23b52bf48c1862a86fa933297c5'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:09.996218 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ebe548dd-2519-4a00-afe0-e1882e4977de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3234a23b52bf48c1862a86fa933297c5'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:10.002954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ebe548dd-2519-4a00-afe0-e1882e4977de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3234a23b52bf48c1862a86fa933297c5'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:10.011806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ebe548dd-2519-4a00-afe0-e1882e4977de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3234a23b52bf48c1862a86fa933297c5'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:10.012887 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ebe548dd-2519-4a00-afe0-e1882e4977de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3234a23b52bf48c1862a86fa933297c5'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:10.018728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1345/2696] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:59:09 2022] DELETE /identity/v3/users/3234a23b52bf48c1862a86fa933297c5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:59:10.299134 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e0f15-ddab-4541-8017-f5d43581a1e4 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:10.300716 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-8d3e0f15-ddab-4541-8017-f5d43581a1e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:10.317127 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e0f15-ddab-4541-8017-f5d43581a1e4 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:10.360029 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:59:10.414891 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ce3be-a342-48f0-976b-0f2f3f3fa7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:10.415578 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ce3be-a342-48f0-976b-0f2f3f3fa7a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:10.415861 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ce3be-a342-48f0-976b-0f2f3f3fa7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:10.416134 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ce3be-a342-48f0-976b-0f2f3f3fa7a6 admin admin] PATH_INFO: `/v3/projects/0f4ab2958efc4806beee8fb71d2a0823` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:10.417348 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ce3be-a342-48f0-976b-0f2f3f3fa7a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f4ab2958efc4806beee8fb71d2a0823)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:10.419354 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ce3be-a342-48f0-976b-0f2f3f3fa7a6 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:10.469782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-ef5ce3be-a342-48f0-976b-0f2f3f3fa7a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f4ab2958efc4806beee8fb71d2a0823', 'request_id': 'req-ef5ce3be-a342-48f0-976b-0f2f3f3fa7a6'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:10.478184 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-ef5ce3be-a342-48f0-976b-0f2f3f3fa7a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f4ab2958efc4806beee8fb71d2a0823', 'request_id': 'req-ef5ce3be-a342-48f0-976b-0f2f3f3fa7a6'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:10.494844 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-ef5ce3be-a342-48f0-976b-0f2f3f3fa7a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f4ab2958efc4806beee8fb71d2a0823', 'request_id': 'req-ef5ce3be-a342-48f0-976b-0f2f3f3fa7a6'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:10.511235 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-ef5ce3be-a342-48f0-976b-0f2f3f3fa7a6 admin admin] The token cache is being invalidate because project 0f4ab2958efc4806beee8fb71d2a0823 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:59:10.511537 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-ef5ce3be-a342-48f0-976b-0f2f3f3fa7a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:10.512194 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-ef5ce3be-a342-48f0-976b-0f2f3f3fa7a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:10.514026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1352/2697] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:59:10 2022] DELETE /identity/v3/projects/0f4ab2958efc4806beee8fb71d2a0823 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:59:10.518746 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95fa43-c0f5-4d26-ab0c-106e6afbdff4 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:10.519798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95fa43-c0f5-4d26-ab0c-106e6afbdff4 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:10.519798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95fa43-c0f5-4d26-ab0c-106e6afbdff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:10.543913 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-8b95fa43-c0f5-4d26-ab0c-106e6afbdff4 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:59:10.545114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-8b95fa43-c0f5-4d26-ab0c-106e6afbdff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:10.562999 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:59:10.612299 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1346/2698] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:59:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:59:10.621828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff8e60-d8b0-4c25-9749-3c717c02d58d None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:10.622198 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff8e60-d8b0-4c25-9749-3c717c02d58d None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:10.622433 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff8e60-d8b0-4c25-9749-3c717c02d58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:10.641890 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-4dff8e60-d8b0-4c25-9749-3c717c02d58d None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:59:10.643124 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4dff8e60-d8b0-4c25-9749-3c717c02d58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:10.651753 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1353/2699] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:59:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:59:10.656064 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95fa43-c0f5-4d26-ab0c-106e6afbdff4 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:10.667853 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95fa43-c0f5-4d26-ab0c-106e6afbdff4 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:10.673069 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab31f7ce-f0bf-45a2-88fc-10316f457e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:10.673665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31f7ce-f0bf-45a2-88fc-10316f457e4f admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:10.673942 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31f7ce-f0bf-45a2-88fc-10316f457e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:10.674214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31f7ce-f0bf-45a2-88fc-10316f457e4f admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:10.674854 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab31f7ce-f0bf-45a2-88fc-10316f457e4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:59:10.675199 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab31f7ce-f0bf-45a2-88fc-10316f457e4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:10.676782 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab31f7ce-f0bf-45a2-88fc-10316f457e4f admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:10.688149 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1347/2700] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:59:10.692860 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff8e60-d8b0-4c25-9749-3c717c02d58d None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:10.702478 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff8e60-d8b0-4c25-9749-3c717c02d58d None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:10.707384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60250dfb-e325-4d8c-a7df-d0849f5710b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:10.708063 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60250dfb-e325-4d8c-a7df-d0849f5710b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:10.708344 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60250dfb-e325-4d8c-a7df-d0849f5710b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:10.708614 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60250dfb-e325-4d8c-a7df-d0849f5710b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:10.709235 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60250dfb-e325-4d8c-a7df-d0849f5710b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:10.711843 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60250dfb-e325-4d8c-a7df-d0849f5710b7 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:10.739914 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1354/2701] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:59:10 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:59:10.744511 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab31f7ce-f0bf-45a2-88fc-10316f457e4f admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:10.754754 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab31f7ce-f0bf-45a2-88fc-10316f457e4f admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:10.811890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e7955-3626-410a-9ca4-47137bc73a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:10.812496 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e7955-3626-410a-9ca4-47137bc73a4a admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:10.812778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e7955-3626-410a-9ca4-47137bc73a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:10.813081 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e7955-3626-410a-9ca4-47137bc73a4a admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:10.813720 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e7955-3626-410a-9ca4-47137bc73a4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:10.815707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e7955-3626-410a-9ca4-47137bc73a4a admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:10.837406 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1348/2702] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:59:10 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:59:10.841622 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60250dfb-e325-4d8c-a7df-d0849f5710b7 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:10.853472 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60250dfb-e325-4d8c-a7df-d0849f5710b7 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:10.858330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8020f9-c07f-45e2-a122-bd15db5094b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:10.858964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8020f9-c07f-45e2-a122-bd15db5094b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:10.859292 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8020f9-c07f-45e2-a122-bd15db5094b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:10.859564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8020f9-c07f-45e2-a122-bd15db5094b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:10.860257 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8020f9-c07f-45e2-a122-bd15db5094b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:10.861844 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8020f9-c07f-45e2-a122-bd15db5094b5 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:10.871813 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1355/2703] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:59:10.875844 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e7955-3626-410a-9ca4-47137bc73a4a admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:10.886444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e7955-3626-410a-9ca4-47137bc73a4a admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:10.891022 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be5e00-3666-4f36-964e-0880bf868776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:10.891647 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be5e00-3666-4f36-964e-0880bf868776 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:10.891926 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be5e00-3666-4f36-964e-0880bf868776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:10.892197 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be5e00-3666-4f36-964e-0880bf868776 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:10.892835 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45be5e00-3666-4f36-964e-0880bf868776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:10.894407 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45be5e00-3666-4f36-964e-0880bf868776 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:10.905645 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1349/2704] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:59:10.909869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8020f9-c07f-45e2-a122-bd15db5094b5 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:10.925161 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8020f9-c07f-45e2-a122-bd15db5094b5 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:10.930748 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48525c8d-5d61-40e7-8ebe-e9697ec7cc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:10.931541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48525c8d-5d61-40e7-8ebe-e9697ec7cc14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:10.931893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48525c8d-5d61-40e7-8ebe-e9697ec7cc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:10.932179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48525c8d-5d61-40e7-8ebe-e9697ec7cc14 admin admin] PATH_INFO: `/v3/projects/1c3b91e098b84d8ebc92196439eb4475/users/7c93f5a87e2c40cf850c12b56518cb45/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:10.947614 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48525c8d-5d61-40e7-8ebe-e9697ec7cc14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c3b91e098b84d8ebc92196439eb4475, user_id=7c93f5a87e2c40cf850c12b56518cb45, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:10.950414 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48525c8d-5d61-40e7-8ebe-e9697ec7cc14 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:10.967104 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1356/2705] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:59:10 2022] PUT /identity/v3/projects/1c3b91e098b84d8ebc92196439eb4475/users/7c93f5a87e2c40cf850c12b56518cb45/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:59:10.978198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24254646-b509-47e9-84b0-c7b1d932ffbd None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:10.978480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24254646-b509-47e9-84b0-c7b1d932ffbd None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:10.978755 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24254646-b509-47e9-84b0-c7b1d932ffbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:11.007189 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-24254646-b509-47e9-84b0-c7b1d932ffbd None None] MFA Rules not processed for user `7c93f5a87e2c40cf850c12b56518cb45`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:59:11.008389 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-24254646-b509-47e9-84b0-c7b1d932ffbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:11.072360 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1350/2706] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:59:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:59:11.082962 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48525c8d-5d61-40e7-8ebe-e9697ec7cc14 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:11.086090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-48525c8d-5d61-40e7-8ebe-e9697ec7cc14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:11.134326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:59:11.147941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48525c8d-5d61-40e7-8ebe-e9697ec7cc14 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:11.152289 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb2ccd-93ec-49c2-9cd5-1201c8735e5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:11.152896 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb2ccd-93ec-49c2-9cd5-1201c8735e5f service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:11.153185 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb2ccd-93ec-49c2-9cd5-1201c8735e5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:11.153459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb2ccd-93ec-49c2-9cd5-1201c8735e5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:11.161820 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb2ccd-93ec-49c2-9cd5-1201c8735e5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:11.164313 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb2ccd-93ec-49c2-9cd5-1201c8735e5f service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:11.173458 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1357/2707] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:59:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:59:21.069489 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:59:21.169311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=7 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:59:22.232823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d4d08-054a-4101-bc7e-09831c14d1db None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:22.233296 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d4d08-054a-4101-bc7e-09831c14d1db None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:22.233400 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d4d08-054a-4101-bc7e-09831c14d1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:22.254045 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-8c5d4d08-054a-4101-bc7e-09831c14d1db None None] MFA Rules not processed for user `a8fb7a9acefd4b418c992661037c2ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:59:22.255195 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-8c5d4d08-054a-4101-bc7e-09831c14d1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:22.308980 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1351/2708] 10.222.0.67 () {58 vars in 967 bytes} [Wed Aug 31 00:59:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:59:22.322104 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb2ccd-93ec-49c2-9cd5-1201c8735e5f service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:22.323551 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-9beb2ccd-93ec-49c2-9cd5-1201c8735e5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:22.393458 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb2ccd-93ec-49c2-9cd5-1201c8735e5f service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:22.399109 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9d4c5-bf18-4896-a17e-db52b5786c87 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd3cfbc8ebd4c5a83bef09efe10bc91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:22.399714 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9d4c5-bf18-4896-a17e-db52b5786c87 service swift] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:22.399992 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9d4c5-bf18-4896-a17e-db52b5786c87 service swift] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:22.400262 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9d4c5-bf18-4896-a17e-db52b5786c87 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:22.408313 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d9d4c5-bf18-4896-a17e-db52b5786c87 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:22.409905 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d9d4c5-bf18-4896-a17e-db52b5786c87 service swift] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:22.418057 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1358/2709] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 31 00:59:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 00:59:22.816606 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d4d08-054a-4101-bc7e-09831c14d1db None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:22.882985 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d4d08-054a-4101-bc7e-09831c14d1db None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:22.887903 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bdc2d-0e9a-451f-b64b-970717e93da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:22.888573 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bdc2d-0e9a-451f-b64b-970717e93da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:22.888852 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bdc2d-0e9a-451f-b64b-970717e93da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:22.889122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bdc2d-0e9a-451f-b64b-970717e93da3 admin admin] PATH_INFO: `/v3/users/7c93f5a87e2c40cf850c12b56518cb45` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:22.890267 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51bdc2d-0e9a-451f-b64b-970717e93da3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c93f5a87e2c40cf850c12b56518cb45)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:22.892207 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51bdc2d-0e9a-451f-b64b-970717e93da3 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:22.942757 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e51bdc2d-0e9a-451f-b64b-970717e93da3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c93f5a87e2c40cf850c12b56518cb45'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:22.956440 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e51bdc2d-0e9a-451f-b64b-970717e93da3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c93f5a87e2c40cf850c12b56518cb45'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:22.964498 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e51bdc2d-0e9a-451f-b64b-970717e93da3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c93f5a87e2c40cf850c12b56518cb45'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:22.985198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e51bdc2d-0e9a-451f-b64b-970717e93da3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c93f5a87e2c40cf850c12b56518cb45'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:22.986077 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e51bdc2d-0e9a-451f-b64b-970717e93da3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c93f5a87e2c40cf850c12b56518cb45'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:23.001665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1352/2710] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:59:22 2022] DELETE /identity/v3/users/7c93f5a87e2c40cf850c12b56518cb45 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:59:23.007754 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5df925-7fa2-4201-b7c0-83b894c996f4 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:23.008431 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5df925-7fa2-4201-b7c0-83b894c996f4 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:23.009019 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5df925-7fa2-4201-b7c0-83b894c996f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:23.032026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-2a5df925-7fa2-4201-b7c0-83b894c996f4 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:59:23.033346 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-2a5df925-7fa2-4201-b7c0-83b894c996f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:23.104245 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1359/2711] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:59:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:59:23.122622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bdc2d-0e9a-451f-b64b-970717e93da3 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:23.125390 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-e51bdc2d-0e9a-451f-b64b-970717e93da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:23.158097 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:59:23.198012 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bdc2d-0e9a-451f-b64b-970717e93da3 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:23.202174 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b0dc9-2d6f-40c7-b67e-2d92c0410219 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:23.202801 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b0dc9-2d6f-40c7-b67e-2d92c0410219 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:23.203091 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b0dc9-2d6f-40c7-b67e-2d92c0410219 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:23.203368 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b0dc9-2d6f-40c7-b67e-2d92c0410219 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:23.214047 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8b0dc9-2d6f-40c7-b67e-2d92c0410219 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:23.215601 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8b0dc9-2d6f-40c7-b67e-2d92c0410219 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:23.227338 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1353/2712] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:59:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:59:23.611985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5df925-7fa2-4201-b7c0-83b894c996f4 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:23.614845 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-2a5df925-7fa2-4201-b7c0-83b894c996f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:23.633377 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5df925-7fa2-4201-b7c0-83b894c996f4 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:23.638435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d7f82-b5f3-4969-a3fa-c374484a5a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:23.639161 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d7f82-b5f3-4969-a3fa-c374484a5a55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:23.639444 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d7f82-b5f3-4969-a3fa-c374484a5a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:23.639805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d7f82-b5f3-4969-a3fa-c374484a5a55 admin admin] PATH_INFO: `/v3/projects/1c3b91e098b84d8ebc92196439eb4475` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:23.641183 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d7f82-b5f3-4969-a3fa-c374484a5a55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c3b91e098b84d8ebc92196439eb4475)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:23.642921 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d7f82-b5f3-4969-a3fa-c374484a5a55 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:23.692006 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-019d7f82-b5f3-4969-a3fa-c374484a5a55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c3b91e098b84d8ebc92196439eb4475', 'request_id': 'req-019d7f82-b5f3-4969-a3fa-c374484a5a55'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:23.698700 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-019d7f82-b5f3-4969-a3fa-c374484a5a55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c3b91e098b84d8ebc92196439eb4475', 'request_id': 'req-019d7f82-b5f3-4969-a3fa-c374484a5a55'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:23.706756 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-019d7f82-b5f3-4969-a3fa-c374484a5a55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c3b91e098b84d8ebc92196439eb4475', 'request_id': 'req-019d7f82-b5f3-4969-a3fa-c374484a5a55'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:23.720818 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-019d7f82-b5f3-4969-a3fa-c374484a5a55 admin admin] The token cache is being invalidate because project 1c3b91e098b84d8ebc92196439eb4475 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:59:23.721207 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-019d7f82-b5f3-4969-a3fa-c374484a5a55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:23.721914 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-019d7f82-b5f3-4969-a3fa-c374484a5a55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:23.723437 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1360/2713] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:59:23 2022] DELETE /identity/v3/projects/1c3b91e098b84d8ebc92196439eb4475 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:59:23.729120 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667bf74-1f09-40f2-a345-436482e94c4a None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:23.729597 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667bf74-1f09-40f2-a345-436482e94c4a None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:23.729789 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667bf74-1f09-40f2-a345-436482e94c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:23.749334 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-e667bf74-1f09-40f2-a345-436482e94c4a None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:59:23.750532 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-e667bf74-1f09-40f2-a345-436482e94c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:23.814050 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1354/2714] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:59:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 00:59:23.823563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c624b5-8269-4b1c-9717-74a86535e40a None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:23.823853 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c624b5-8269-4b1c-9717-74a86535e40a None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:23.824129 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c624b5-8269-4b1c-9717-74a86535e40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:23.845407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-13c624b5-8269-4b1c-9717-74a86535e40a None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:59:23.846610 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-13c624b5-8269-4b1c-9717-74a86535e40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:23.855966 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1361/2715] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:59:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:59:23.860454 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e667bf74-1f09-40f2-a345-436482e94c4a None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:23.872993 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e667bf74-1f09-40f2-a345-436482e94c4a None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:23.877774 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a93b8f-9f9c-44b8-ba16-10d8b1ad96e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:23.878383 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a93b8f-9f9c-44b8-ba16-10d8b1ad96e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:23.878668 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a93b8f-9f9c-44b8-ba16-10d8b1ad96e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:23.878943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a93b8f-9f9c-44b8-ba16-10d8b1ad96e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:23.879637 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a93b8f-9f9c-44b8-ba16-10d8b1ad96e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 00:59:23.879968 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a93b8f-9f9c-44b8-ba16-10d8b1ad96e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:23.881778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a93b8f-9f9c-44b8-ba16-10d8b1ad96e1 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:23.894638 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1355/2716] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 00:59:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 00:59:23.899743 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c624b5-8269-4b1c-9717-74a86535e40a None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:23.909115 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c624b5-8269-4b1c-9717-74a86535e40a None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:23.914927 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42185da2-d332-4483-b675-18d945ab8f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:23.915572 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42185da2-d332-4483-b675-18d945ab8f17 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:23.915854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42185da2-d332-4483-b675-18d945ab8f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:23.916128 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42185da2-d332-4483-b675-18d945ab8f17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:23.916767 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42185da2-d332-4483-b675-18d945ab8f17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:23.918678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42185da2-d332-4483-b675-18d945ab8f17 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:23.924061 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:59:23.951919 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1362/2717] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 00:59:23 2022] POST /identity/v3/projects => generated 399 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 00:59:23.956206 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a93b8f-9f9c-44b8-ba16-10d8b1ad96e1 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:23.967410 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a93b8f-9f9c-44b8-ba16-10d8b1ad96e1 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:24.040332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b05ba-5230-4f6d-95ce-622dd84b03f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:24.040946 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b05ba-5230-4f6d-95ce-622dd84b03f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:24.041230 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b05ba-5230-4f6d-95ce-622dd84b03f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:24.041507 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b05ba-5230-4f6d-95ce-622dd84b03f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:24.042139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b05ba-5230-4f6d-95ce-622dd84b03f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:24.044106 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b05ba-5230-4f6d-95ce-622dd84b03f1 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:24.067254 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1356/2718] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 00:59:23 2022] POST /identity/v3/users => generated 300 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 00:59:24.072121 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42185da2-d332-4483-b675-18d945ab8f17 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:24.082578 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42185da2-d332-4483-b675-18d945ab8f17 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:24.088050 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1caac9-2de2-4de0-bc77-87d65fbb33e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:24.088745 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1caac9-2de2-4de0-bc77-87d65fbb33e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:24.089089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1caac9-2de2-4de0-bc77-87d65fbb33e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:24.089423 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1caac9-2de2-4de0-bc77-87d65fbb33e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:24.090143 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1caac9-2de2-4de0-bc77-87d65fbb33e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:24.091835 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1caac9-2de2-4de0-bc77-87d65fbb33e3 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:24.105399 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1363/2719] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:59:24.106868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b05ba-5230-4f6d-95ce-622dd84b03f1 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:24.117459 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b05ba-5230-4f6d-95ce-622dd84b03f1 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:24.122257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aae1fc-5dfb-44a9-a90a-d703e38e1107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:24.122824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aae1fc-5dfb-44a9-a90a-d703e38e1107 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:24.123135 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aae1fc-5dfb-44a9-a90a-d703e38e1107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:24.123409 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aae1fc-5dfb-44a9-a90a-d703e38e1107 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:24.124044 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aae1fc-5dfb-44a9-a90a-d703e38e1107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:24.125780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aae1fc-5dfb-44a9-a90a-d703e38e1107 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:24.138508 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1357/2720] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 00:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 00:59:24.142556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1caac9-2de2-4de0-bc77-87d65fbb33e3 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:24.153664 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1caac9-2de2-4de0-bc77-87d65fbb33e3 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:24.159566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32618698-5ed0-4d76-a47d-82f56b5db824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:24.160230 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32618698-5ed0-4d76-a47d-82f56b5db824 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:24.160537 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32618698-5ed0-4d76-a47d-82f56b5db824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:24.160814 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32618698-5ed0-4d76-a47d-82f56b5db824 admin admin] PATH_INFO: `/v3/projects/5b0da8a9fad44fc89461e37adf274e1d/users/ad229246c5594fd7826bd12793027523/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:24.180482 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32618698-5ed0-4d76-a47d-82f56b5db824 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b0da8a9fad44fc89461e37adf274e1d, user_id=ad229246c5594fd7826bd12793027523, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:24.183225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32618698-5ed0-4d76-a47d-82f56b5db824 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:24.196445 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1364/2721] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 00:59:24 2022] PUT /identity/v3/projects/5b0da8a9fad44fc89461e37adf274e1d/users/ad229246c5594fd7826bd12793027523/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:59:24.206823 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56dcbdd-646b-4954-a01b-16f6285a1128 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:24.207146 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56dcbdd-646b-4954-a01b-16f6285a1128 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:24.207424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56dcbdd-646b-4954-a01b-16f6285a1128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:24.236266 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-b56dcbdd-646b-4954-a01b-16f6285a1128 None None] MFA Rules not processed for user `ad229246c5594fd7826bd12793027523`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:59:24.237480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-b56dcbdd-646b-4954-a01b-16f6285a1128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:24.257532 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:59:24.305895 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1358/2722] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:59:24 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:59:24.317316 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32618698-5ed0-4d76-a47d-82f56b5db824 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:24.319312 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-32618698-5ed0-4d76-a47d-82f56b5db824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:24.377860 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32618698-5ed0-4d76-a47d-82f56b5db824 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:24.381689 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc49705-4fe0-4772-9a35-ca1692c8c544 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:24.382540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc49705-4fe0-4772-9a35-ca1692c8c544 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:24.382824 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc49705-4fe0-4772-9a35-ca1692c8c544 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:24.383129 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc49705-4fe0-4772-9a35-ca1692c8c544 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:24.391471 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc49705-4fe0-4772-9a35-ca1692c8c544 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:24.392902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc49705-4fe0-4772-9a35-ca1692c8c544 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:24.402484 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1365/2723] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:59:24 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:59:34.302596 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=17 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:59:34.398536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:59:34.574181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56dcbdd-646b-4954-a01b-16f6285a1128 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:34.641491 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56dcbdd-646b-4954-a01b-16f6285a1128 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:34.646119 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772429ec-581e-42dd-bc93-c1c00e9ce1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:34.646981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772429ec-581e-42dd-bc93-c1c00e9ce1ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:34.647143 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772429ec-581e-42dd-bc93-c1c00e9ce1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:34.647378 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772429ec-581e-42dd-bc93-c1c00e9ce1ca admin admin] PATH_INFO: `/v3/users/ad229246c5594fd7826bd12793027523` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:34.648936 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772429ec-581e-42dd-bc93-c1c00e9ce1ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad229246c5594fd7826bd12793027523)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:34.650799 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772429ec-581e-42dd-bc93-c1c00e9ce1ca admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:34.703261 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-772429ec-581e-42dd-bc93-c1c00e9ce1ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad229246c5594fd7826bd12793027523'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:34.715343 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-772429ec-581e-42dd-bc93-c1c00e9ce1ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad229246c5594fd7826bd12793027523'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:34.722640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-772429ec-581e-42dd-bc93-c1c00e9ce1ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad229246c5594fd7826bd12793027523'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:34.730743 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-772429ec-581e-42dd-bc93-c1c00e9ce1ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad229246c5594fd7826bd12793027523'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:34.731391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-772429ec-581e-42dd-bc93-c1c00e9ce1ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad229246c5594fd7826bd12793027523'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:34.737275 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1359/2724] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 00:59:34 2022] DELETE /identity/v3/users/ad229246c5594fd7826bd12793027523 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 00:59:34.742928 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601a65d-9058-4199-9add-5b57df7edaa4 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:34.743253 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601a65d-9058-4199-9add-5b57df7edaa4 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:34.743531 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601a65d-9058-4199-9add-5b57df7edaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:34.764604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-8601a65d-9058-4199-9add-5b57df7edaa4 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 00:59:34.765774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-8601a65d-9058-4199-9add-5b57df7edaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:34.831430 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1366/2725] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 00:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 00:59:34.841697 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772429ec-581e-42dd-bc93-c1c00e9ce1ca admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:34.843129 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-772429ec-581e-42dd-bc93-c1c00e9ce1ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:34.902948 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772429ec-581e-42dd-bc93-c1c00e9ce1ca admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:34.906565 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5774cdd-c8b0-438a-9e7e-678d574e343e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:34.907216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5774cdd-c8b0-438a-9e7e-678d574e343e service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:34.907533 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5774cdd-c8b0-438a-9e7e-678d574e343e service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:34.907807 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5774cdd-c8b0-438a-9e7e-678d574e343e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:34.917829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5774cdd-c8b0-438a-9e7e-678d574e343e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:34.919286 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5774cdd-c8b0-438a-9e7e-678d574e343e service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:34.929019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1360/2726] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:59:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:59:35.479693 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601a65d-9058-4199-9add-5b57df7edaa4 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:35.481153 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-8601a65d-9058-4199-9add-5b57df7edaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:35.493103 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601a65d-9058-4199-9add-5b57df7edaa4 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:35.498149 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4e9ca-23ce-48eb-a7f9-16d3e28a342a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:35.498795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4e9ca-23ce-48eb-a7f9-16d3e28a342a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:35.499101 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4e9ca-23ce-48eb-a7f9-16d3e28a342a admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:35.499378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4e9ca-23ce-48eb-a7f9-16d3e28a342a admin admin] PATH_INFO: `/v3/projects/5b0da8a9fad44fc89461e37adf274e1d` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:35.500712 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c4e9ca-23ce-48eb-a7f9-16d3e28a342a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b0da8a9fad44fc89461e37adf274e1d)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:35.503240 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c4e9ca-23ce-48eb-a7f9-16d3e28a342a admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:35.553911 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-90c4e9ca-23ce-48eb-a7f9-16d3e28a342a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b0da8a9fad44fc89461e37adf274e1d', 'request_id': 'req-90c4e9ca-23ce-48eb-a7f9-16d3e28a342a'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:35.559378 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-90c4e9ca-23ce-48eb-a7f9-16d3e28a342a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b0da8a9fad44fc89461e37adf274e1d', 'request_id': 'req-90c4e9ca-23ce-48eb-a7f9-16d3e28a342a'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:35.565493 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-90c4e9ca-23ce-48eb-a7f9-16d3e28a342a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b0da8a9fad44fc89461e37adf274e1d', 'request_id': 'req-90c4e9ca-23ce-48eb-a7f9-16d3e28a342a'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:35.578555 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-90c4e9ca-23ce-48eb-a7f9-16d3e28a342a admin admin] The token cache is being invalidate because project 5b0da8a9fad44fc89461e37adf274e1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 00:59:35.578847 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-90c4e9ca-23ce-48eb-a7f9-16d3e28a342a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:35.579505 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-90c4e9ca-23ce-48eb-a7f9-16d3e28a342a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 00:59:35.580993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1367/2727] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 00:59:35 2022] DELETE /identity/v3/projects/5b0da8a9fad44fc89461e37adf274e1d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 00:59:38.933819 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5774cdd-c8b0-438a-9e7e-678d574e343e service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 00:59:38.935246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-a5774cdd-c8b0-438a-9e7e-678d574e343e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:38.946771 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5774cdd-c8b0-438a-9e7e-678d574e343e service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 00:59:38.964440 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:59:38.999797 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03defb-b7ea-4314-9a24-d048a9729598 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 00:59:39.000405 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03defb-b7ea-4314-9a24-d048a9729598 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 00:59:39.000688 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03defb-b7ea-4314-9a24-d048a9729598 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 00:59:39.000959 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03defb-b7ea-4314-9a24-d048a9729598 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 00:59:39.002715 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-5f03defb-b7ea-4314-9a24-d048a9729598 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 00:59:39.091417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03defb-b7ea-4314-9a24-d048a9729598 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 00:59:39.092852 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03defb-b7ea-4314-9a24-d048a9729598 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 00:59:39.104139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1361/2728] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 00:59:38 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 00:59:45.575133 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 00:59:49.098763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=40 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:00:10.933749 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4e9ca-23ce-48eb-a7f9-16d3e28a342a admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:10.935207 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-90c4e9ca-23ce-48eb-a7f9-16d3e28a342a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:00:11.028129 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4e9ca-23ce-48eb-a7f9-16d3e28a342a admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:11.033205 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42092e84-b13b-4221-a08a-d95fb11a0a3f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdacdcb2451742938164fdd44cdbaebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:11.033812 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42092e84-b13b-4221-a08a-d95fb11a0a3f service placement] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:11.034092 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42092e84-b13b-4221-a08a-d95fb11a0a3f service placement] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:11.034362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42092e84-b13b-4221-a08a-d95fb11a0a3f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:11.036218 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-42092e84-b13b-4221-a08a-d95fb11a0a3f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:00:11.048699 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42092e84-b13b-4221-a08a-d95fb11a0a3f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:11.050307 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42092e84-b13b-4221-a08a-d95fb11a0a3f service placement] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:11.060445 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1368/2729] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:00:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:00:21.056115 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=41 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:00:42.395862 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03defb-b7ea-4314-9a24-d048a9729598 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:42.397266 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-5f03defb-b7ea-4314-9a24-d048a9729598 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:00:42.408078 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03defb-b7ea-4314-9a24-d048a9729598 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:42.411700 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49656fac-d716-454b-aeae-c9821b6a7e89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:42.412297 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49656fac-d716-454b-aeae-c9821b6a7e89 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:42.412576 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49656fac-d716-454b-aeae-c9821b6a7e89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:42.412847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49656fac-d716-454b-aeae-c9821b6a7e89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:42.414516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-49656fac-d716-454b-aeae-c9821b6a7e89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:00:42.478274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49656fac-d716-454b-aeae-c9821b6a7e89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:42.479843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49656fac-d716-454b-aeae-c9821b6a7e89 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:42.489701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1362/2730] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:00:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:00:45.695070 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42092e84-b13b-4221-a08a-d95fb11a0a3f service placement] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:45.696807 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-42092e84-b13b-4221-a08a-d95fb11a0a3f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:00:45.714302 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42092e84-b13b-4221-a08a-d95fb11a0a3f service placement] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:45.719940 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8e3b0-b175-4418-a61e-c98d9e063eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:45.720603 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8e3b0-b175-4418-a61e-c98d9e063eb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:45.720879 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8e3b0-b175-4418-a61e-c98d9e063eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:45.721145 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8e3b0-b175-4418-a61e-c98d9e063eb8 admin admin] PATH_INFO: `/v3/users/7d7582f9cdd646f39404f7adc99ad9ac` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:45.722445 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8e3b0-b175-4418-a61e-c98d9e063eb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d7582f9cdd646f39404f7adc99ad9ac)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:45.724393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8e3b0-b175-4418-a61e-c98d9e063eb8 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:45.771058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c4f8e3b0-b175-4418-a61e-c98d9e063eb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d7582f9cdd646f39404f7adc99ad9ac'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:45.783069 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c4f8e3b0-b175-4418-a61e-c98d9e063eb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d7582f9cdd646f39404f7adc99ad9ac'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:45.792886 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c4f8e3b0-b175-4418-a61e-c98d9e063eb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d7582f9cdd646f39404f7adc99ad9ac'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:45.793625 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c4f8e3b0-b175-4418-a61e-c98d9e063eb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d7582f9cdd646f39404f7adc99ad9ac'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:45.794229 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c4f8e3b0-b175-4418-a61e-c98d9e063eb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d7582f9cdd646f39404f7adc99ad9ac'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:45.811739 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1369/2731] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 01:00:45 2022] DELETE /identity/v3/users/7d7582f9cdd646f39404f7adc99ad9ac => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:00:45.986361 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49656fac-d716-454b-aeae-c9821b6a7e89 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:45.997646 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49656fac-d716-454b-aeae-c9821b6a7e89 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:46.054995 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41278d7f-415e-49af-8217-5b787bab86e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:46.055701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41278d7f-415e-49af-8217-5b787bab86e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:46.056094 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41278d7f-415e-49af-8217-5b787bab86e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:46.056463 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41278d7f-415e-49af-8217-5b787bab86e1 admin admin] PATH_INFO: `/v3/projects/5694aa9e83804fe19311f97ed6af1152` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:46.058189 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41278d7f-415e-49af-8217-5b787bab86e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5694aa9e83804fe19311f97ed6af1152)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:46.060106 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41278d7f-415e-49af-8217-5b787bab86e1 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:46.105294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-41278d7f-415e-49af-8217-5b787bab86e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5694aa9e83804fe19311f97ed6af1152', 'request_id': 'req-41278d7f-415e-49af-8217-5b787bab86e1'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:46.112626 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-41278d7f-415e-49af-8217-5b787bab86e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5694aa9e83804fe19311f97ed6af1152', 'request_id': 'req-41278d7f-415e-49af-8217-5b787bab86e1'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:46.118135 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-41278d7f-415e-49af-8217-5b787bab86e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5694aa9e83804fe19311f97ed6af1152', 'request_id': 'req-41278d7f-415e-49af-8217-5b787bab86e1'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:46.129501 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-41278d7f-415e-49af-8217-5b787bab86e1 admin admin] The token cache is being invalidate because project 5694aa9e83804fe19311f97ed6af1152 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:00:46.129784 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-41278d7f-415e-49af-8217-5b787bab86e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:46.130533 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-41278d7f-415e-49af-8217-5b787bab86e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:46.132235 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1363/2732] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 01:00:45 2022] DELETE /identity/v3/projects/5694aa9e83804fe19311f97ed6af1152 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:00:46.137130 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5559dc-1c62-4cde-b4ca-2a98fc692888 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:46.137416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5559dc-1c62-4cde-b4ca-2a98fc692888 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:46.137688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5559dc-1c62-4cde-b4ca-2a98fc692888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:46.159515 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-ce5559dc-1c62-4cde-b4ca-2a98fc692888 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:00:46.160696 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-ce5559dc-1c62-4cde-b4ca-2a98fc692888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:00:46.220414 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1370/2733] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:00:46.228528 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348fdc0-43df-4e63-bc95-c45f6a1247b7 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:46.228995 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348fdc0-43df-4e63-bc95-c45f6a1247b7 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:46.229451 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348fdc0-43df-4e63-bc95-c45f6a1247b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:46.250182 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-1348fdc0-43df-4e63-bc95-c45f6a1247b7 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:00:46.251432 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-1348fdc0-43df-4e63-bc95-c45f6a1247b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:00:46.259313 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1364/2734] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:00:46.264168 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5559dc-1c62-4cde-b4ca-2a98fc692888 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:46.273911 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5559dc-1c62-4cde-b4ca-2a98fc692888 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:46.278864 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b084d8-c014-4915-98b4-f88de0ef2c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:46.279503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b084d8-c014-4915-98b4-f88de0ef2c5f admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:46.279785 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b084d8-c014-4915-98b4-f88de0ef2c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:46.280059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b084d8-c014-4915-98b4-f88de0ef2c5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:46.280702 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b084d8-c014-4915-98b4-f88de0ef2c5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:00:46.281006 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b084d8-c014-4915-98b4-f88de0ef2c5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:46.282601 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b084d8-c014-4915-98b4-f88de0ef2c5f admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:46.294552 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1371/2735] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 01:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:00:46.298728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348fdc0-43df-4e63-bc95-c45f6a1247b7 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:46.308660 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348fdc0-43df-4e63-bc95-c45f6a1247b7 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:46.313363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b58ccd-6e48-4f91-baa1-184bd6d0946a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:46.313957 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b58ccd-6e48-4f91-baa1-184bd6d0946a admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:46.314238 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b58ccd-6e48-4f91-baa1-184bd6d0946a admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:46.314509 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b58ccd-6e48-4f91-baa1-184bd6d0946a admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:46.315259 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b58ccd-6e48-4f91-baa1-184bd6d0946a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:46.317166 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b58ccd-6e48-4f91-baa1-184bd6d0946a admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:46.333197 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:00:46.347304 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1365/2736] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 01:00:46 2022] POST /identity/v3/projects => generated 407 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:00:46.351404 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b084d8-c014-4915-98b4-f88de0ef2c5f admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:46.361764 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b084d8-c014-4915-98b4-f88de0ef2c5f admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:46.419871 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601a74c-f8b6-45a5-96dc-b058716c3c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:46.420480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601a74c-f8b6-45a5-96dc-b058716c3c1c admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:46.420762 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601a74c-f8b6-45a5-96dc-b058716c3c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:46.421034 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601a74c-f8b6-45a5-96dc-b058716c3c1c admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:46.421669 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2601a74c-f8b6-45a5-96dc-b058716c3c1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:46.423622 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2601a74c-f8b6-45a5-96dc-b058716c3c1c admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:46.431503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:00:46.446204 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1372/2737] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 01:00:46 2022] POST /identity/v3/users => generated 304 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:00:46.450902 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b58ccd-6e48-4f91-baa1-184bd6d0946a admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:46.460996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b58ccd-6e48-4f91-baa1-184bd6d0946a admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:46.466027 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600c75c-2f99-4dc2-b12b-512feb3133ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:46.466623 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600c75c-2f99-4dc2-b12b-512feb3133ab admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:46.466898 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600c75c-2f99-4dc2-b12b-512feb3133ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:46.467233 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600c75c-2f99-4dc2-b12b-512feb3133ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:46.467865 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0600c75c-2f99-4dc2-b12b-512feb3133ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:46.469714 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0600c75c-2f99-4dc2-b12b-512feb3133ab admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:46.479744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1366/2738] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:00:46.483841 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601a74c-f8b6-45a5-96dc-b058716c3c1c admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:46.493658 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601a74c-f8b6-45a5-96dc-b058716c3c1c admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:46.498739 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c6e0eb-5642-4290-b126-4ff781188b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:46.499363 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6e0eb-5642-4290-b126-4ff781188b43 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:46.499754 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6e0eb-5642-4290-b126-4ff781188b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:46.500035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6e0eb-5642-4290-b126-4ff781188b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:46.500673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c6e0eb-5642-4290-b126-4ff781188b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:46.502261 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c6e0eb-5642-4290-b126-4ff781188b43 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:46.513170 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1373/2739] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:00:46.517246 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600c75c-2f99-4dc2-b12b-512feb3133ab admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:46.527377 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600c75c-2f99-4dc2-b12b-512feb3133ab admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:46.531938 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469ee2b-6651-452b-bf2f-35a9bcd49d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:46.532578 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469ee2b-6651-452b-bf2f-35a9bcd49d25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:46.532852 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469ee2b-6651-452b-bf2f-35a9bcd49d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:46.533122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469ee2b-6651-452b-bf2f-35a9bcd49d25 admin admin] PATH_INFO: `/v3/projects/43cb6093841d4556b706ce37987c085a/users/6b821a31a5f54ff5a9aaf71dc62b9d41/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:46.547640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9469ee2b-6651-452b-bf2f-35a9bcd49d25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43cb6093841d4556b706ce37987c085a, user_id=6b821a31a5f54ff5a9aaf71dc62b9d41, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:46.550240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9469ee2b-6651-452b-bf2f-35a9bcd49d25 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:46.563977 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1367/2740] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 01:00:46 2022] PUT /identity/v3/projects/43cb6093841d4556b706ce37987c085a/users/6b821a31a5f54ff5a9aaf71dc62b9d41/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:00:46.576508 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4e1df-3079-44ed-866c-181d4d8ad3f8 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:46.576802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4e1df-3079-44ed-866c-181d4d8ad3f8 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:46.577095 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4e1df-3079-44ed-866c-181d4d8ad3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:46.605415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-ddf4e1df-3079-44ed-866c-181d4d8ad3f8 None None] MFA Rules not processed for user `6b821a31a5f54ff5a9aaf71dc62b9d41`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:00:46.606620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-ddf4e1df-3079-44ed-866c-181d4d8ad3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:00:46.673667 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1374/2741] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:00:46 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:00:46.678440 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469ee2b-6651-452b-bf2f-35a9bcd49d25 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:46.740157 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469ee2b-6651-452b-bf2f-35a9bcd49d25 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:46.745360 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f49b7dc-8a77-47a5-a291-2a582669d138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:46.745950 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f49b7dc-8a77-47a5-a291-2a582669d138 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:46.746224 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f49b7dc-8a77-47a5-a291-2a582669d138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:46.746488 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f49b7dc-8a77-47a5-a291-2a582669d138 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:46.747159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f49b7dc-8a77-47a5-a291-2a582669d138 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:46.749343 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f49b7dc-8a77-47a5-a291-2a582669d138 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:46.775225 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1368/2742] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 01:00:46 2022] POST /identity/v3/projects => generated 407 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:00:46.779577 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf4e1df-3079-44ed-866c-181d4d8ad3f8 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:46.790093 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf4e1df-3079-44ed-866c-181d4d8ad3f8 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:46.856718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e5a59-79c9-406e-ba10-d805e17ae419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:46.857331 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e5a59-79c9-406e-ba10-d805e17ae419 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:46.857617 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e5a59-79c9-406e-ba10-d805e17ae419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:46.857893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e5a59-79c9-406e-ba10-d805e17ae419 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:46.858520 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e5a59-79c9-406e-ba10-d805e17ae419 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:46.860476 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e5a59-79c9-406e-ba10-d805e17ae419 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:46.882547 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1375/2743] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 01:00:46 2022] POST /identity/v3/users => generated 310 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:00:46.887023 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f49b7dc-8a77-47a5-a291-2a582669d138 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:46.897849 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f49b7dc-8a77-47a5-a291-2a582669d138 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:46.903174 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c077ff-5091-4ab0-a253-ed30962cffc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:46.903766 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c077ff-5091-4ab0-a253-ed30962cffc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:46.904043 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c077ff-5091-4ab0-a253-ed30962cffc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:46.904310 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c077ff-5091-4ab0-a253-ed30962cffc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:46.904938 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c077ff-5091-4ab0-a253-ed30962cffc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:46.906533 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c077ff-5091-4ab0-a253-ed30962cffc5 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:46.917901 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1369/2744] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:00:46.922125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e5a59-79c9-406e-ba10-d805e17ae419 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:46.932054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e5a59-79c9-406e-ba10-d805e17ae419 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:46.936596 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd849e-be69-42cb-aa35-630405ebb252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:46.937241 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd849e-be69-42cb-aa35-630405ebb252 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:46.937520 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd849e-be69-42cb-aa35-630405ebb252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:46.937792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd849e-be69-42cb-aa35-630405ebb252 admin admin] PATH_INFO: `/v3/projects/8a7f956d64984bcfac293aac9ab4a66b/users/e69872459f974338bcee6efa9bef459b/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:46.952184 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dd849e-be69-42cb-aa35-630405ebb252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a7f956d64984bcfac293aac9ab4a66b, user_id=e69872459f974338bcee6efa9bef459b, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:46.954871 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dd849e-be69-42cb-aa35-630405ebb252 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:46.967560 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1376/2745] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 01:00:46 2022] PUT /identity/v3/projects/8a7f956d64984bcfac293aac9ab4a66b/users/e69872459f974338bcee6efa9bef459b/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:00:46.971947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c077ff-5091-4ab0-a253-ed30962cffc5 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:46.983164 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c077ff-5091-4ab0-a253-ed30962cffc5 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:47.041778 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c4197-2e5b-4958-b293-f554ff45cd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:47.042397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c4197-2e5b-4958-b293-f554ff45cd8a admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:47.042653 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c4197-2e5b-4958-b293-f554ff45cd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:47.042920 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c4197-2e5b-4958-b293-f554ff45cd8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:47.043605 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c4197-2e5b-4958-b293-f554ff45cd8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:47.045199 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c4197-2e5b-4958-b293-f554ff45cd8a admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:47.055129 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1370/2746] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:00:47.059409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd849e-be69-42cb-aa35-630405ebb252 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:47.069979 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd849e-be69-42cb-aa35-630405ebb252 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:47.075130 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c957462-da21-4c27-aaed-995abdb1a09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:47.075777 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c957462-da21-4c27-aaed-995abdb1a09a admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:47.076058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c957462-da21-4c27-aaed-995abdb1a09a admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:47.076337 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c957462-da21-4c27-aaed-995abdb1a09a admin admin] PATH_INFO: `/v3/projects/8a7f956d64984bcfac293aac9ab4a66b/users/e69872459f974338bcee6efa9bef459b/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:47.078270 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c957462-da21-4c27-aaed-995abdb1a09a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a7f956d64984bcfac293aac9ab4a66b, user_id=e69872459f974338bcee6efa9bef459b, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:47.080965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c957462-da21-4c27-aaed-995abdb1a09a admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:47.092157 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1377/2747] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 01:00:47 2022] PUT /identity/v3/projects/8a7f956d64984bcfac293aac9ab4a66b/users/e69872459f974338bcee6efa9bef459b/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:00:47.159565 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc713313-170f-4732-ad35-c24957a2bab9 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:47.159745 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc713313-170f-4732-ad35-c24957a2bab9 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:47.160031 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc713313-170f-4732-ad35-c24957a2bab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:47.188222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-fc713313-170f-4732-ad35-c24957a2bab9 None None] MFA Rules not processed for user `e69872459f974338bcee6efa9bef459b`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:00:47.189425 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-fc713313-170f-4732-ad35-c24957a2bab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:00:47.208359 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:00:47.259472 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1371/2748] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:00:47 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:00:47.270994 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c957462-da21-4c27-aaed-995abdb1a09a admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:47.272719 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-1c957462-da21-4c27-aaed-995abdb1a09a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:00:47.295755 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:00:47.333528 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c957462-da21-4c27-aaed-995abdb1a09a admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:47.337205 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87fc11-5081-4ae0-85d2-73abeb0766ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:47.337814 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87fc11-5081-4ae0-85d2-73abeb0766ad service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:47.338221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87fc11-5081-4ae0-85d2-73abeb0766ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:47.338495 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87fc11-5081-4ae0-85d2-73abeb0766ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:47.347526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb87fc11-5081-4ae0-85d2-73abeb0766ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:47.348963 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb87fc11-5081-4ae0-85d2-73abeb0766ad service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:47.358860 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1378/2749] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:00:47 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:00:47.509005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc713313-170f-4732-ad35-c24957a2bab9 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:47.519655 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc713313-170f-4732-ad35-c24957a2bab9 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:47.523153 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b8680-1c0b-4725-aa16-4dacd84b2dcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:47.523743 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b8680-1c0b-4725-aa16-4dacd84b2dcb service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:47.524061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b8680-1c0b-4725-aa16-4dacd84b2dcb service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:47.524335 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b8680-1c0b-4725-aa16-4dacd84b2dcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:47.582867 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b8680-1c0b-4725-aa16-4dacd84b2dcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:47.584520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b8680-1c0b-4725-aa16-4dacd84b2dcb service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:47.594292 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1372/2750] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:00:47 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:00:56.973403 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87fc11-5081-4ae0-85d2-73abeb0766ad service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:57.035543 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87fc11-5081-4ae0-85d2-73abeb0766ad service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:57.040082 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f0b95-a21d-4f1b-82a6-c05eda24ebd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:57.040899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f0b95-a21d-4f1b-82a6-c05eda24ebd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:57.041241 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f0b95-a21d-4f1b-82a6-c05eda24ebd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:57.041519 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f0b95-a21d-4f1b-82a6-c05eda24ebd0 admin admin] PATH_INFO: `/v3/users/6b821a31a5f54ff5a9aaf71dc62b9d41` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:57.043213 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f0b95-a21d-4f1b-82a6-c05eda24ebd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b821a31a5f54ff5a9aaf71dc62b9d41)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:57.045219 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f0b95-a21d-4f1b-82a6-c05eda24ebd0 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:57.097158 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-7f9f0b95-a21d-4f1b-82a6-c05eda24ebd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b821a31a5f54ff5a9aaf71dc62b9d41'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:57.104092 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-7f9f0b95-a21d-4f1b-82a6-c05eda24ebd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b821a31a5f54ff5a9aaf71dc62b9d41'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:57.111740 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-7f9f0b95-a21d-4f1b-82a6-c05eda24ebd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b821a31a5f54ff5a9aaf71dc62b9d41'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:57.112429 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-7f9f0b95-a21d-4f1b-82a6-c05eda24ebd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b821a31a5f54ff5a9aaf71dc62b9d41'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:57.113037 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-7f9f0b95-a21d-4f1b-82a6-c05eda24ebd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b821a31a5f54ff5a9aaf71dc62b9d41'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:57.130890 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1379/2751] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 01:00:56 2022] DELETE /identity/v3/users/6b821a31a5f54ff5a9aaf71dc62b9d41 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:00:57.135529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf9265-729e-458e-9da2-8f8e34a0896d None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:57.136012 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf9265-729e-458e-9da2-8f8e34a0896d None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:57.136191 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf9265-729e-458e-9da2-8f8e34a0896d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:57.155810 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-a2cf9265-729e-458e-9da2-8f8e34a0896d None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:00:57.157011 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-a2cf9265-729e-458e-9da2-8f8e34a0896d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:00:57.218565 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1373/2752] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:00:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:00:57.228634 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f0b95-a21d-4f1b-82a6-c05eda24ebd0 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:57.230080 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-7f9f0b95-a21d-4f1b-82a6-c05eda24ebd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:00:57.275614 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:00:57.290330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f0b95-a21d-4f1b-82a6-c05eda24ebd0 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:57.294329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d90736-6f5f-4ace-9cb0-3c652b99abf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:57.294961 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d90736-6f5f-4ace-9cb0-3c652b99abf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:57.295271 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d90736-6f5f-4ace-9cb0-3c652b99abf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:57.295543 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d90736-6f5f-4ace-9cb0-3c652b99abf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:57.304504 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d90736-6f5f-4ace-9cb0-3c652b99abf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:57.305965 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d90736-6f5f-4ace-9cb0-3c652b99abf3 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:57.315670 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1380/2753] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:00:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:00:57.586669 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf9265-729e-458e-9da2-8f8e34a0896d None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:57.588431 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-a2cf9265-729e-458e-9da2-8f8e34a0896d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:00:57.602436 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf9265-729e-458e-9da2-8f8e34a0896d None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:57.607535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16fdb62-aeb6-4b10-a98d-af8b287742ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:57.608186 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16fdb62-aeb6-4b10-a98d-af8b287742ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:57.608466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16fdb62-aeb6-4b10-a98d-af8b287742ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:57.608782 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16fdb62-aeb6-4b10-a98d-af8b287742ce admin admin] PATH_INFO: `/v3/projects/43cb6093841d4556b706ce37987c085a` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:57.610104 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16fdb62-aeb6-4b10-a98d-af8b287742ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=43cb6093841d4556b706ce37987c085a)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:57.612527 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16fdb62-aeb6-4b10-a98d-af8b287742ce admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:57.660820 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e16fdb62-aeb6-4b10-a98d-af8b287742ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43cb6093841d4556b706ce37987c085a', 'request_id': 'req-e16fdb62-aeb6-4b10-a98d-af8b287742ce'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:57.662479 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:00:57.669003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e16fdb62-aeb6-4b10-a98d-af8b287742ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43cb6093841d4556b706ce37987c085a', 'request_id': 'req-e16fdb62-aeb6-4b10-a98d-af8b287742ce'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:57.674693 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e16fdb62-aeb6-4b10-a98d-af8b287742ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43cb6093841d4556b706ce37987c085a', 'request_id': 'req-e16fdb62-aeb6-4b10-a98d-af8b287742ce'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:57.686503 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e16fdb62-aeb6-4b10-a98d-af8b287742ce admin admin] The token cache is being invalidate because project 43cb6093841d4556b706ce37987c085a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:00:57.686792 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e16fdb62-aeb6-4b10-a98d-af8b287742ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:57.687465 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e16fdb62-aeb6-4b10-a98d-af8b287742ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:57.689002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1374/2754] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 01:00:57 2022] DELETE /identity/v3/projects/43cb6093841d4556b706ce37987c085a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:00:57.693066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d90736-6f5f-4ace-9cb0-3c652b99abf3 service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:57.703293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d90736-6f5f-4ace-9cb0-3c652b99abf3 service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:57.759199 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219e89d-3173-4647-91bf-a60d63d9f2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:57.759831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219e89d-3173-4647-91bf-a60d63d9f2a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:57.760119 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219e89d-3173-4647-91bf-a60d63d9f2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:57.760520 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219e89d-3173-4647-91bf-a60d63d9f2a8 admin admin] PATH_INFO: `/v3/users/e69872459f974338bcee6efa9bef459b` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:57.762079 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7219e89d-3173-4647-91bf-a60d63d9f2a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e69872459f974338bcee6efa9bef459b)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:57.764048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7219e89d-3173-4647-91bf-a60d63d9f2a8 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:57.811136 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-7219e89d-3173-4647-91bf-a60d63d9f2a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e69872459f974338bcee6efa9bef459b'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:57.818428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-7219e89d-3173-4647-91bf-a60d63d9f2a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e69872459f974338bcee6efa9bef459b'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:57.825519 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-7219e89d-3173-4647-91bf-a60d63d9f2a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e69872459f974338bcee6efa9bef459b'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:57.826179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-7219e89d-3173-4647-91bf-a60d63d9f2a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e69872459f974338bcee6efa9bef459b'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:57.826791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-7219e89d-3173-4647-91bf-a60d63d9f2a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e69872459f974338bcee6efa9bef459b'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:57.844421 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1381/2755] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 01:00:57 2022] DELETE /identity/v3/users/e69872459f974338bcee6efa9bef459b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:00:58.116531 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16fdb62-aeb6-4b10-a98d-af8b287742ce admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:58.118068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-e16fdb62-aeb6-4b10-a98d-af8b287742ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:00:58.133503 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16fdb62-aeb6-4b10-a98d-af8b287742ce admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:58.194081 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a3935-22a0-440d-b922-01cb290bd2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:58.194735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a3935-22a0-440d-b922-01cb290bd2d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:58.194997 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a3935-22a0-440d-b922-01cb290bd2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:58.195303 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a3935-22a0-440d-b922-01cb290bd2d9 admin admin] PATH_INFO: `/v3/projects/8a7f956d64984bcfac293aac9ab4a66b` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:58.196397 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a3935-22a0-440d-b922-01cb290bd2d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a7f956d64984bcfac293aac9ab4a66b)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:58.198274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a3935-22a0-440d-b922-01cb290bd2d9 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:58.245713 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-fe5a3935-22a0-440d-b922-01cb290bd2d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a7f956d64984bcfac293aac9ab4a66b', 'request_id': 'req-fe5a3935-22a0-440d-b922-01cb290bd2d9'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:58.253108 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-fe5a3935-22a0-440d-b922-01cb290bd2d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a7f956d64984bcfac293aac9ab4a66b', 'request_id': 'req-fe5a3935-22a0-440d-b922-01cb290bd2d9'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:58.259549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-fe5a3935-22a0-440d-b922-01cb290bd2d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a7f956d64984bcfac293aac9ab4a66b', 'request_id': 'req-fe5a3935-22a0-440d-b922-01cb290bd2d9'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:58.271239 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-fe5a3935-22a0-440d-b922-01cb290bd2d9 admin admin] The token cache is being invalidate because project 8a7f956d64984bcfac293aac9ab4a66b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:00:58.271526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-fe5a3935-22a0-440d-b922-01cb290bd2d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:58.272190 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-fe5a3935-22a0-440d-b922-01cb290bd2d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:58.273656 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1375/2756] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 01:00:58 2022] DELETE /identity/v3/projects/8a7f956d64984bcfac293aac9ab4a66b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:00:58.279586 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc091d-af50-4a57-8593-25da04ef112a None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:58.280064 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc091d-af50-4a57-8593-25da04ef112a None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:58.280189 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc091d-af50-4a57-8593-25da04ef112a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:58.301819 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-50dc091d-af50-4a57-8593-25da04ef112a None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:00:58.303023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-50dc091d-af50-4a57-8593-25da04ef112a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:00:58.361298 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1382/2757] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:00:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:00:58.369543 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9694922-545a-45f0-974c-c8164eb2b279 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:58.369773 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9694922-545a-45f0-974c-c8164eb2b279 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:58.370051 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9694922-545a-45f0-974c-c8164eb2b279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:58.392424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-d9694922-545a-45f0-974c-c8164eb2b279 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:00:58.393727 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-d9694922-545a-45f0-974c-c8164eb2b279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:00:58.406530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1376/2758] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:00:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:00:58.410970 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc091d-af50-4a57-8593-25da04ef112a None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:58.417201 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:00:58.422291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc091d-af50-4a57-8593-25da04ef112a None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:58.428480 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ddf84-4718-4696-aff6-c4832645eb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:58.429195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ddf84-4718-4696-aff6-c4832645eb1f admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:58.429554 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ddf84-4718-4696-aff6-c4832645eb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:58.429901 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ddf84-4718-4696-aff6-c4832645eb1f admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:58.430694 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784ddf84-4718-4696-aff6-c4832645eb1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:00:58.431290 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784ddf84-4718-4696-aff6-c4832645eb1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:58.432782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784ddf84-4718-4696-aff6-c4832645eb1f admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:58.445606 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1383/2759] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 01:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:00:58.450570 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9694922-545a-45f0-974c-c8164eb2b279 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:58.459853 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9694922-545a-45f0-974c-c8164eb2b279 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:58.464979 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59c13a-25e1-4e83-85e2-3c955d7c8dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:58.465567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59c13a-25e1-4e83-85e2-3c955d7c8dec admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:58.465844 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59c13a-25e1-4e83-85e2-3c955d7c8dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:58.466114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59c13a-25e1-4e83-85e2-3c955d7c8dec admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:58.466743 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59c13a-25e1-4e83-85e2-3c955d7c8dec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:58.468705 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59c13a-25e1-4e83-85e2-3c955d7c8dec admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:58.497061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1377/2760] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 01:00:58 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:00:58.501182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ddf84-4718-4696-aff6-c4832645eb1f admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:58.511550 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ddf84-4718-4696-aff6-c4832645eb1f admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:58.567917 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661e10b-22d8-4f5b-9806-fa147f725cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:58.568562 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661e10b-22d8-4f5b-9806-fa147f725cdc admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:58.568953 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661e10b-22d8-4f5b-9806-fa147f725cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:58.569301 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661e10b-22d8-4f5b-9806-fa147f725cdc admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:58.569999 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9661e10b-22d8-4f5b-9806-fa147f725cdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:58.572030 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9661e10b-22d8-4f5b-9806-fa147f725cdc admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:58.593412 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1384/2761] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 01:00:58 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:00:58.597787 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59c13a-25e1-4e83-85e2-3c955d7c8dec admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:58.609027 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59c13a-25e1-4e83-85e2-3c955d7c8dec admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:58.614701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd163fc9-f0fa-49c6-8cc9-d8cd80b1cc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:58.615324 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd163fc9-f0fa-49c6-8cc9-d8cd80b1cc67 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:58.615600 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd163fc9-f0fa-49c6-8cc9-d8cd80b1cc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:58.615867 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd163fc9-f0fa-49c6-8cc9-d8cd80b1cc67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:58.616504 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd163fc9-f0fa-49c6-8cc9-d8cd80b1cc67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:58.618086 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd163fc9-f0fa-49c6-8cc9-d8cd80b1cc67 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:58.629028 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1378/2762] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:00:58.632952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661e10b-22d8-4f5b-9806-fa147f725cdc admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:58.643108 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661e10b-22d8-4f5b-9806-fa147f725cdc admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:58.647819 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9e777-97b2-4493-8a75-6b6e9dc717b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:58.648426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9e777-97b2-4493-8a75-6b6e9dc717b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:58.648696 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9e777-97b2-4493-8a75-6b6e9dc717b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:58.648967 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9e777-97b2-4493-8a75-6b6e9dc717b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:58.649604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9e777-97b2-4493-8a75-6b6e9dc717b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:58.651243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9e777-97b2-4493-8a75-6b6e9dc717b1 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:58.661960 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1385/2763] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:00:58.666669 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd163fc9-f0fa-49c6-8cc9-d8cd80b1cc67 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:58.677165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd163fc9-f0fa-49c6-8cc9-d8cd80b1cc67 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:58.681939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e940cf-314e-4b86-90d4-287b58f304f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:58.682615 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e940cf-314e-4b86-90d4-287b58f304f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:58.683505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e940cf-314e-4b86-90d4-287b58f304f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:58.683782 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e940cf-314e-4b86-90d4-287b58f304f5 admin admin] PATH_INFO: `/v3/projects/a37777f57e964ed3ac885890b4b67165/users/528716d23172429b8cb3928bb5ae59ae/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:58.697801 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e940cf-314e-4b86-90d4-287b58f304f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a37777f57e964ed3ac885890b4b67165, user_id=528716d23172429b8cb3928bb5ae59ae, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:58.700476 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e940cf-314e-4b86-90d4-287b58f304f5 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:58.713566 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1379/2764] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 01:00:58 2022] PUT /identity/v3/projects/a37777f57e964ed3ac885890b4b67165/users/528716d23172429b8cb3928bb5ae59ae/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:00:58.725429 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03ae8b-b13c-4d10-a649-4d5d683d1eb1 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:58.725717 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03ae8b-b13c-4d10-a649-4d5d683d1eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:58.725993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03ae8b-b13c-4d10-a649-4d5d683d1eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:58.754854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-4a03ae8b-b13c-4d10-a649-4d5d683d1eb1 None None] MFA Rules not processed for user `528716d23172429b8cb3928bb5ae59ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:00:58.756097 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4a03ae8b-b13c-4d10-a649-4d5d683d1eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:00:58.820794 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1386/2765] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:00:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:00:58.832480 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e940cf-314e-4b86-90d4-287b58f304f5 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:58.834227 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-75e940cf-314e-4b86-90d4-287b58f304f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:00:58.841654 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:00:58.895639 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e940cf-314e-4b86-90d4-287b58f304f5 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:58.899279 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723ba13-a101-41da-bd41-cfb674a13344 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:58.899884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723ba13-a101-41da-bd41-cfb674a13344 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:58.900164 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723ba13-a101-41da-bd41-cfb674a13344 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:58.900436 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723ba13-a101-41da-bd41-cfb674a13344 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:58.909705 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0723ba13-a101-41da-bd41-cfb674a13344 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:58.911144 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0723ba13-a101-41da-bd41-cfb674a13344 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:58.920951 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1380/2766] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:00:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:00:58.944243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03ae8b-b13c-4d10-a649-4d5d683d1eb1 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:59.015679 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03ae8b-b13c-4d10-a649-4d5d683d1eb1 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:59.020348 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86220c6-c9f9-4be4-8c16-b8fe63194f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:59.020975 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86220c6-c9f9-4be4-8c16-b8fe63194f22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:59.021258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86220c6-c9f9-4be4-8c16-b8fe63194f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:59.021530 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86220c6-c9f9-4be4-8c16-b8fe63194f22 admin admin] PATH_INFO: `/v3/users/528716d23172429b8cb3928bb5ae59ae` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:59.022658 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86220c6-c9f9-4be4-8c16-b8fe63194f22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=528716d23172429b8cb3928bb5ae59ae)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:59.024631 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86220c6-c9f9-4be4-8c16-b8fe63194f22 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:59.026091 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:00:59.200109 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-b86220c6-c9f9-4be4-8c16-b8fe63194f22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '528716d23172429b8cb3928bb5ae59ae'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:59.208126 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-b86220c6-c9f9-4be4-8c16-b8fe63194f22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '528716d23172429b8cb3928bb5ae59ae'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:59.216567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-b86220c6-c9f9-4be4-8c16-b8fe63194f22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '528716d23172429b8cb3928bb5ae59ae'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:59.217583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-b86220c6-c9f9-4be4-8c16-b8fe63194f22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '528716d23172429b8cb3928bb5ae59ae'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:59.218595 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-b86220c6-c9f9-4be4-8c16-b8fe63194f22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '528716d23172429b8cb3928bb5ae59ae'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:59.244393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1387/2767] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 01:00:58 2022] DELETE /identity/v3/users/528716d23172429b8cb3928bb5ae59ae => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:00:59.249501 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aadb770-dc53-42ba-b368-46b5372ebf1e None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:59.250023 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aadb770-dc53-42ba-b368-46b5372ebf1e None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:59.250111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aadb770-dc53-42ba-b368-46b5372ebf1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:59.277359 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-9aadb770-dc53-42ba-b368-46b5372ebf1e None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:00:59.279215 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-9aadb770-dc53-42ba-b368-46b5372ebf1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:00:59.352314 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1381/2768] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:00:59.362207 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86220c6-c9f9-4be4-8c16-b8fe63194f22 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:59.363623 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b86220c6-c9f9-4be4-8c16-b8fe63194f22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:00:59.420909 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86220c6-c9f9-4be4-8c16-b8fe63194f22 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:59.424691 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde632d-671a-4506-80d0-e941db8f82f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:59.425291 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde632d-671a-4506-80d0-e941db8f82f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:59.425571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde632d-671a-4506-80d0-e941db8f82f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:59.425843 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde632d-671a-4506-80d0-e941db8f82f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:59.434727 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde632d-671a-4506-80d0-e941db8f82f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:59.436187 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde632d-671a-4506-80d0-e941db8f82f0 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:59.445605 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1388/2769] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:00:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:00:59.713877 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aadb770-dc53-42ba-b368-46b5372ebf1e None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:59.716066 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-9aadb770-dc53-42ba-b368-46b5372ebf1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:00:59.751403 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aadb770-dc53-42ba-b368-46b5372ebf1e None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:59.757940 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85850a33-0038-4ff2-bd53-a4f129ac0e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:59.758590 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85850a33-0038-4ff2-bd53-a4f129ac0e5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:59.758906 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85850a33-0038-4ff2-bd53-a4f129ac0e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:59.759221 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85850a33-0038-4ff2-bd53-a4f129ac0e5f admin admin] PATH_INFO: `/v3/projects/a37777f57e964ed3ac885890b4b67165` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:59.760373 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85850a33-0038-4ff2-bd53-a4f129ac0e5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a37777f57e964ed3ac885890b4b67165)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:59.762415 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85850a33-0038-4ff2-bd53-a4f129ac0e5f admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:59.809144 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-85850a33-0038-4ff2-bd53-a4f129ac0e5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a37777f57e964ed3ac885890b4b67165', 'request_id': 'req-85850a33-0038-4ff2-bd53-a4f129ac0e5f'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:59.816659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-85850a33-0038-4ff2-bd53-a4f129ac0e5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a37777f57e964ed3ac885890b4b67165', 'request_id': 'req-85850a33-0038-4ff2-bd53-a4f129ac0e5f'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:59.822215 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-85850a33-0038-4ff2-bd53-a4f129ac0e5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a37777f57e964ed3ac885890b4b67165', 'request_id': 'req-85850a33-0038-4ff2-bd53-a4f129ac0e5f'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:59.834015 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-85850a33-0038-4ff2-bd53-a4f129ac0e5f admin admin] The token cache is being invalidate because project a37777f57e964ed3ac885890b4b67165 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:00:59.834304 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-85850a33-0038-4ff2-bd53-a4f129ac0e5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:59.834939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-85850a33-0038-4ff2-bd53-a4f129ac0e5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:00:59.836477 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1382/2770] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 01:00:59 2022] DELETE /identity/v3/projects/a37777f57e964ed3ac885890b4b67165 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:00:59.841858 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094dece-befc-4b7f-8649-f0d991439957 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:59.842137 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094dece-befc-4b7f-8649-f0d991439957 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:59.842407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094dece-befc-4b7f-8649-f0d991439957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:59.862563 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-3094dece-befc-4b7f-8649-f0d991439957 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:00:59.863772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-3094dece-befc-4b7f-8649-f0d991439957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:00:59.921059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1389/2771] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:00:59.928714 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a7491-e1fc-4630-8ada-364a4963b016 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:59.928998 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a7491-e1fc-4630-8ada-364a4963b016 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:59.929265 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a7491-e1fc-4630-8ada-364a4963b016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:59.949036 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-aa2a7491-e1fc-4630-8ada-364a4963b016 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:00:59.950475 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-aa2a7491-e1fc-4630-8ada-364a4963b016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:00:59.958298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1383/2772] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:00:59.962584 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094dece-befc-4b7f-8649-f0d991439957 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:00:59.973495 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094dece-befc-4b7f-8649-f0d991439957 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:00:59.978279 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df50131-3695-4cc2-878d-d7bdd4b9bb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:00:59.978890 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df50131-3695-4cc2-878d-d7bdd4b9bb40 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:00:59.979358 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df50131-3695-4cc2-878d-d7bdd4b9bb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:00:59.979713 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df50131-3695-4cc2-878d-d7bdd4b9bb40 admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:00:59.980687 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df50131-3695-4cc2-878d-d7bdd4b9bb40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:00:59.980998 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df50131-3695-4cc2-878d-d7bdd4b9bb40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:00:59.982617 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df50131-3695-4cc2-878d-d7bdd4b9bb40 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:00:59.994863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1390/2773] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 01:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:00:59.999220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a7491-e1fc-4630-8ada-364a4963b016 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:01:00.004776 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:01:00.008825 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a7491-e1fc-4630-8ada-364a4963b016 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:01:00.015383 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547d94f-6930-42b3-9d9b-00074a959568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:01:00.016043 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547d94f-6930-42b3-9d9b-00074a959568 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:00.016955 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547d94f-6930-42b3-9d9b-00074a959568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:00.016955 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547d94f-6930-42b3-9d9b-00074a959568 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:00.017810 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9547d94f-6930-42b3-9d9b-00074a959568 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:01:00.021513 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9547d94f-6930-42b3-9d9b-00074a959568 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:01:00.049808 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1384/2774] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 01:00:59 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:01:00.053960 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df50131-3695-4cc2-878d-d7bdd4b9bb40 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:01:00.064072 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df50131-3695-4cc2-878d-d7bdd4b9bb40 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:01:00.068577 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:01:00.121434 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de759430-d95d-46ca-930f-9d2e6d3a9c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:01:00.122035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de759430-d95d-46ca-930f-9d2e6d3a9c69 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:00.122314 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de759430-d95d-46ca-930f-9d2e6d3a9c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:00.122584 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de759430-d95d-46ca-930f-9d2e6d3a9c69 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:00.123240 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de759430-d95d-46ca-930f-9d2e6d3a9c69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:01:00.125143 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de759430-d95d-46ca-930f-9d2e6d3a9c69 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:01:00.147879 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1391/2775] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 01:01:00 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:01:00.152686 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547d94f-6930-42b3-9d9b-00074a959568 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:01:00.163507 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547d94f-6930-42b3-9d9b-00074a959568 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:01:00.168457 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31492081-bf05-4c73-8c3a-66d44645b7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:01:00.169084 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31492081-bf05-4c73-8c3a-66d44645b7d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:00.169361 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31492081-bf05-4c73-8c3a-66d44645b7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:00.169630 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31492081-bf05-4c73-8c3a-66d44645b7d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:00.170259 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31492081-bf05-4c73-8c3a-66d44645b7d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:01:00.171863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31492081-bf05-4c73-8c3a-66d44645b7d7 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:01:00.181810 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1385/2776] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:01:00.185954 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de759430-d95d-46ca-930f-9d2e6d3a9c69 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:01:00.195740 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de759430-d95d-46ca-930f-9d2e6d3a9c69 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:01:00.201033 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb2ee2-6026-4d65-959c-569f122a4da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:01:00.201682 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb2ee2-6026-4d65-959c-569f122a4da8 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:00.202032 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb2ee2-6026-4d65-959c-569f122a4da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:00.202349 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb2ee2-6026-4d65-959c-569f122a4da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:00.202993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb2ee2-6026-4d65-959c-569f122a4da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:01:00.204599 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb2ee2-6026-4d65-959c-569f122a4da8 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:01:00.215985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1392/2777] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:01:00.220068 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31492081-bf05-4c73-8c3a-66d44645b7d7 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:01:00.230903 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31492081-bf05-4c73-8c3a-66d44645b7d7 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:01:00.235603 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b1fe1-9462-44c5-9a2e-3a6981e3d790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:01:00.236249 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b1fe1-9462-44c5-9a2e-3a6981e3d790 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:00.236533 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b1fe1-9462-44c5-9a2e-3a6981e3d790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:00.236812 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b1fe1-9462-44c5-9a2e-3a6981e3d790 admin admin] PATH_INFO: `/v3/projects/f1c00918d4ba41aa9816f723fdbe6761/users/94b56b28f6d945f18fb5053b17dec543/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:00.251938 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b1fe1-9462-44c5-9a2e-3a6981e3d790 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1c00918d4ba41aa9816f723fdbe6761, user_id=94b56b28f6d945f18fb5053b17dec543, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:01:00.254589 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b1fe1-9462-44c5-9a2e-3a6981e3d790 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:01:00.268530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1386/2778] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 01:01:00 2022] PUT /identity/v3/projects/f1c00918d4ba41aa9816f723fdbe6761/users/94b56b28f6d945f18fb5053b17dec543/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:01:00.282940 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db205429-23b5-4a6b-b140-1b1cd218dc29 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:00.283251 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db205429-23b5-4a6b-b140-1b1cd218dc29 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:00.283524 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db205429-23b5-4a6b-b140-1b1cd218dc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:00.311952 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-db205429-23b5-4a6b-b140-1b1cd218dc29 None None] MFA Rules not processed for user `94b56b28f6d945f18fb5053b17dec543`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:01:00.313223 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-db205429-23b5-4a6b-b140-1b1cd218dc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:01:00.380127 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1393/2779] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:01:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:01:00.393131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b1fe1-9462-44c5-9a2e-3a6981e3d790 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:01:00.394843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-902b1fe1-9462-44c5-9a2e-3a6981e3d790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:01:00.475371 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b1fe1-9462-44c5-9a2e-3a6981e3d790 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:01:00.479113 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea5ed1-e58e-4ec9-af6a-536c8bf9f573 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55714a21c1cb4f2b949603230243ed02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:01:00.479761 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea5ed1-e58e-4ec9-af6a-536c8bf9f573 service glance] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:00.480647 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea5ed1-e58e-4ec9-af6a-536c8bf9f573 service glance] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:00.481155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea5ed1-e58e-4ec9-af6a-536c8bf9f573 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:00.489937 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea5ed1-e58e-4ec9-af6a-536c8bf9f573 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:01:00.491418 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea5ed1-e58e-4ec9-af6a-536c8bf9f573 service glance] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:01:00.501434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1387/2780] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:01:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:01:04.124179 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e42a58-240b-4508-93bd-51f5782791d0 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:04.124696 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e42a58-240b-4508-93bd-51f5782791d0 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:04.124696 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e42a58-240b-4508-93bd-51f5782791d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:04.143958 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-56e42a58-240b-4508-93bd-51f5782791d0 None None] MFA Rules not processed for user `26826ddf92124c95835a5356e13040bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:01:04.145010 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-56e42a58-240b-4508-93bd-51f5782791d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:01:04.204262 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1394/2781] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:01:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:01:10.497120 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=55,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:01:14.196799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=87,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:01:49.785103 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea5ed1-e58e-4ec9-af6a-536c8bf9f573 service glance] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:01:49.847353 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea5ed1-e58e-4ec9-af6a-536c8bf9f573 service glance] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:01:49.851839 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68437b0b-7eae-4b83-a2d6-a4cdfd84726e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:01:49.852466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68437b0b-7eae-4b83-a2d6-a4cdfd84726e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:49.852742 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68437b0b-7eae-4b83-a2d6-a4cdfd84726e admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:49.853013 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68437b0b-7eae-4b83-a2d6-a4cdfd84726e admin admin] PATH_INFO: `/v3/users/94b56b28f6d945f18fb5053b17dec543` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:49.854128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68437b0b-7eae-4b83-a2d6-a4cdfd84726e admin admin] RBAC: Authorizing `identity:delete_user(user_id=94b56b28f6d945f18fb5053b17dec543)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:01:49.856937 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68437b0b-7eae-4b83-a2d6-a4cdfd84726e admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:01:49.902956 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-68437b0b-7eae-4b83-a2d6-a4cdfd84726e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94b56b28f6d945f18fb5053b17dec543'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:01:49.914951 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-68437b0b-7eae-4b83-a2d6-a4cdfd84726e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94b56b28f6d945f18fb5053b17dec543'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:01:49.921872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-68437b0b-7eae-4b83-a2d6-a4cdfd84726e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94b56b28f6d945f18fb5053b17dec543'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:01:49.929290 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-68437b0b-7eae-4b83-a2d6-a4cdfd84726e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94b56b28f6d945f18fb5053b17dec543'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:01:49.929904 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-68437b0b-7eae-4b83-a2d6-a4cdfd84726e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94b56b28f6d945f18fb5053b17dec543'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:01:49.935862 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1388/2782] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 01:01:49 2022] DELETE /identity/v3/users/94b56b28f6d945f18fb5053b17dec543 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:01:49.940381 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b63370-e163-4654-a1f3-bd4073d70ef4 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:49.940886 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b63370-e163-4654-a1f3-bd4073d70ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:49.940996 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b63370-e163-4654-a1f3-bd4073d70ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:49.960320 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-a5b63370-e163-4654-a1f3-bd4073d70ef4 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:01:49.961514 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a5b63370-e163-4654-a1f3-bd4073d70ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:01:50.020810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1395/2783] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:01:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:01:50.031657 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68437b0b-7eae-4b83-a2d6-a4cdfd84726e admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:01:50.033164 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-68437b0b-7eae-4b83-a2d6-a4cdfd84726e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:01:50.089260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68437b0b-7eae-4b83-a2d6-a4cdfd84726e admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:01:50.093885 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae5672-ed11-4ebf-904f-017081acf743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:01:50.094884 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae5672-ed11-4ebf-904f-017081acf743 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:50.095195 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae5672-ed11-4ebf-904f-017081acf743 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:50.095466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae5672-ed11-4ebf-904f-017081acf743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:50.104563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae5672-ed11-4ebf-904f-017081acf743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:01:50.106007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae5672-ed11-4ebf-904f-017081acf743 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:01:50.115639 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1389/2784] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:01:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:01:50.398691 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b63370-e163-4654-a1f3-bd4073d70ef4 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:01:50.400229 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-a5b63370-e163-4654-a1f3-bd4073d70ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:01:50.411986 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b63370-e163-4654-a1f3-bd4073d70ef4 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:01:50.416568 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6391771-678c-46ae-8040-6e3a1c7fee31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:01:50.417368 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6391771-678c-46ae-8040-6e3a1c7fee31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:50.417693 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6391771-678c-46ae-8040-6e3a1c7fee31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:50.417964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6391771-678c-46ae-8040-6e3a1c7fee31 admin admin] PATH_INFO: `/v3/projects/f1c00918d4ba41aa9816f723fdbe6761` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:50.419197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6391771-678c-46ae-8040-6e3a1c7fee31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1c00918d4ba41aa9816f723fdbe6761)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:01:50.421108 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6391771-678c-46ae-8040-6e3a1c7fee31 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:01:50.468784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d6391771-678c-46ae-8040-6e3a1c7fee31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1c00918d4ba41aa9816f723fdbe6761', 'request_id': 'req-d6391771-678c-46ae-8040-6e3a1c7fee31'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:01:50.474679 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d6391771-678c-46ae-8040-6e3a1c7fee31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1c00918d4ba41aa9816f723fdbe6761', 'request_id': 'req-d6391771-678c-46ae-8040-6e3a1c7fee31'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:01:50.480955 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d6391771-678c-46ae-8040-6e3a1c7fee31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1c00918d4ba41aa9816f723fdbe6761', 'request_id': 'req-d6391771-678c-46ae-8040-6e3a1c7fee31'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:01:50.493449 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d6391771-678c-46ae-8040-6e3a1c7fee31 admin admin] The token cache is being invalidate because project f1c00918d4ba41aa9816f723fdbe6761 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:01:50.493743 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d6391771-678c-46ae-8040-6e3a1c7fee31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:01:50.494361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-d6391771-678c-46ae-8040-6e3a1c7fee31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:01:50.496647 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1396/2785] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 01:01:50 2022] DELETE /identity/v3/projects/f1c00918d4ba41aa9816f723fdbe6761 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:01:50.501473 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03093e-dad4-4125-8924-0c33e025b388 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:50.501763 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03093e-dad4-4125-8924-0c33e025b388 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:50.502063 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03093e-dad4-4125-8924-0c33e025b388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:50.523667 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-df03093e-dad4-4125-8924-0c33e025b388 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:01:50.524860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-df03093e-dad4-4125-8924-0c33e025b388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:01:50.541048 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:01:50.584479 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1390/2786] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:01:50.592261 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aad1ab-9b7c-4105-8093-08bc5142525f None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:50.592569 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aad1ab-9b7c-4105-8093-08bc5142525f None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:50.592847 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aad1ab-9b7c-4105-8093-08bc5142525f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:50.612398 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-b6aad1ab-9b7c-4105-8093-08bc5142525f None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:01:50.613597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b6aad1ab-9b7c-4105-8093-08bc5142525f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:01:50.623115 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1397/2787] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:01:50.627319 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03093e-dad4-4125-8924-0c33e025b388 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:01:50.638319 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03093e-dad4-4125-8924-0c33e025b388 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:01:50.642995 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03512a13-68cf-45ec-bde1-4a1379729edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:01:50.643625 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03512a13-68cf-45ec-bde1-4a1379729edd admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:50.643906 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03512a13-68cf-45ec-bde1-4a1379729edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:50.644178 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03512a13-68cf-45ec-bde1-4a1379729edd admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:50.644819 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03512a13-68cf-45ec-bde1-4a1379729edd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:01:50.645122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03512a13-68cf-45ec-bde1-4a1379729edd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:01:50.646705 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03512a13-68cf-45ec-bde1-4a1379729edd admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:01:50.658339 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1391/2788] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 01:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:01:50.662592 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aad1ab-9b7c-4105-8093-08bc5142525f None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:01:50.672207 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aad1ab-9b7c-4105-8093-08bc5142525f None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:01:50.676866 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a9dea-f740-4f1b-81b7-985c5c2ed782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:01:50.677451 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a9dea-f740-4f1b-81b7-985c5c2ed782 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:50.677723 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a9dea-f740-4f1b-81b7-985c5c2ed782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:50.677985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a9dea-f740-4f1b-81b7-985c5c2ed782 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:50.678600 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242a9dea-f740-4f1b-81b7-985c5c2ed782 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:01:50.680504 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242a9dea-f740-4f1b-81b7-985c5c2ed782 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:01:50.707672 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1398/2789] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 01:01:50 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:01:50.712266 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03512a13-68cf-45ec-bde1-4a1379729edd admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:01:50.723776 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03512a13-68cf-45ec-bde1-4a1379729edd admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:01:50.783476 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b897dd4-9cdc-4e54-a0b7-d6c7d7edd752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:01:50.784082 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b897dd4-9cdc-4e54-a0b7-d6c7d7edd752 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:50.784363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b897dd4-9cdc-4e54-a0b7-d6c7d7edd752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:50.784636 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b897dd4-9cdc-4e54-a0b7-d6c7d7edd752 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:50.785277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b897dd4-9cdc-4e54-a0b7-d6c7d7edd752 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:01:50.787234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b897dd4-9cdc-4e54-a0b7-d6c7d7edd752 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:01:50.809863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1392/2790] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 01:01:50 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:01:50.814270 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a9dea-f740-4f1b-81b7-985c5c2ed782 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:01:50.824887 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a9dea-f740-4f1b-81b7-985c5c2ed782 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:01:50.829529 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ada43-bf98-4768-bf77-872a4385ace9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:01:50.830130 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ada43-bf98-4768-bf77-872a4385ace9 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:50.830409 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ada43-bf98-4768-bf77-872a4385ace9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:50.830681 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ada43-bf98-4768-bf77-872a4385ace9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:50.831383 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ada43-bf98-4768-bf77-872a4385ace9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:01:50.832971 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ada43-bf98-4768-bf77-872a4385ace9 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:01:50.843015 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1399/2791] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:01:50.847453 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b897dd4-9cdc-4e54-a0b7-d6c7d7edd752 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:01:50.858267 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b897dd4-9cdc-4e54-a0b7-d6c7d7edd752 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:01:50.863119 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98cd362-6be6-4bc0-9ee6-0a7580868a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:01:50.863714 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cd362-6be6-4bc0-9ee6-0a7580868a80 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:50.863996 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cd362-6be6-4bc0-9ee6-0a7580868a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:50.864268 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cd362-6be6-4bc0-9ee6-0a7580868a80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:50.864904 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98cd362-6be6-4bc0-9ee6-0a7580868a80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:01:50.866485 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98cd362-6be6-4bc0-9ee6-0a7580868a80 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:01:50.877550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1393/2792] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:01:50.881790 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ada43-bf98-4768-bf77-872a4385ace9 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:01:50.891681 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ada43-bf98-4768-bf77-872a4385ace9 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:01:50.896272 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6d848-6bd4-4950-83bf-a0f311939e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:01:50.896854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6d848-6bd4-4950-83bf-a0f311939e09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:50.897130 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6d848-6bd4-4950-83bf-a0f311939e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:50.897400 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6d848-6bd4-4950-83bf-a0f311939e09 admin admin] PATH_INFO: `/v3/projects/0a4fbdb025bd4ceb9b47f7a91c998862/users/c8a08be3b6434f39b48f397e624c860c/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:50.911523 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb6d848-6bd4-4950-83bf-a0f311939e09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a4fbdb025bd4ceb9b47f7a91c998862, user_id=c8a08be3b6434f39b48f397e624c860c, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:01:50.914150 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb6d848-6bd4-4950-83bf-a0f311939e09 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:01:50.927363 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1400/2793] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 01:01:50 2022] PUT /identity/v3/projects/0a4fbdb025bd4ceb9b47f7a91c998862/users/c8a08be3b6434f39b48f397e624c860c/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:01:50.938214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bbe41-d5e7-43d2-91a0-01c8b85cabc5 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:50.938525 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bbe41-d5e7-43d2-91a0-01c8b85cabc5 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:50.938802 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bbe41-d5e7-43d2-91a0-01c8b85cabc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:50.966369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-aa0bbe41-d5e7-43d2-91a0-01c8b85cabc5 None None] MFA Rules not processed for user `c8a08be3b6434f39b48f397e624c860c`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:01:50.967602 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-aa0bbe41-d5e7-43d2-91a0-01c8b85cabc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:01:51.031619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1394/2794] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:01:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:01:51.043666 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6d848-6bd4-4950-83bf-a0f311939e09 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:01:51.045388 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-9fb6d848-6bd4-4950-83bf-a0f311939e09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:01:51.066829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:01:51.102954 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6d848-6bd4-4950-83bf-a0f311939e09 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:01:51.107288 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51876cd6-ca89-41c6-9663-2777112354fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:01:51.107883 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51876cd6-ca89-41c6-9663-2777112354fd service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:51.108162 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51876cd6-ca89-41c6-9663-2777112354fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:51.108431 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51876cd6-ca89-41c6-9663-2777112354fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:51.116938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51876cd6-ca89-41c6-9663-2777112354fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:01:51.118359 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51876cd6-ca89-41c6-9663-2777112354fd service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:01:51.126985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1401/2795] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:01:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:01:53.669615 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bbe41-d5e7-43d2-91a0-01c8b85cabc5 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:01:53.688850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:01:53.734981 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bbe41-d5e7-43d2-91a0-01c8b85cabc5 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:01:53.739938 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97bd14-8903-49ea-9058-51fdfe5567fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:01:53.740567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97bd14-8903-49ea-9058-51fdfe5567fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:53.740866 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97bd14-8903-49ea-9058-51fdfe5567fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:53.741134 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97bd14-8903-49ea-9058-51fdfe5567fd admin admin] PATH_INFO: `/v3/users/c8a08be3b6434f39b48f397e624c860c` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:53.742272 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97bd14-8903-49ea-9058-51fdfe5567fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8a08be3b6434f39b48f397e624c860c)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:01:53.744218 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97bd14-8903-49ea-9058-51fdfe5567fd admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:01:53.790231 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ea97bd14-8903-49ea-9058-51fdfe5567fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8a08be3b6434f39b48f397e624c860c'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:01:53.802524 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ea97bd14-8903-49ea-9058-51fdfe5567fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8a08be3b6434f39b48f397e624c860c'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:01:53.809982 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ea97bd14-8903-49ea-9058-51fdfe5567fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8a08be3b6434f39b48f397e624c860c'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:01:53.818366 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ea97bd14-8903-49ea-9058-51fdfe5567fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8a08be3b6434f39b48f397e624c860c'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:01:53.819117 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ea97bd14-8903-49ea-9058-51fdfe5567fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8a08be3b6434f39b48f397e624c860c'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:01:53.825381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1395/2796] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 01:01:53 2022] DELETE /identity/v3/users/c8a08be3b6434f39b48f397e624c860c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:01:53.829968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c36043-a4dd-4cd8-9ddd-17f265ced284 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:53.830425 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c36043-a4dd-4cd8-9ddd-17f265ced284 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:53.830530 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c36043-a4dd-4cd8-9ddd-17f265ced284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:53.849605 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-e7c36043-a4dd-4cd8-9ddd-17f265ced284 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:01:53.850720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e7c36043-a4dd-4cd8-9ddd-17f265ced284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:01:53.911515 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1402/2797] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:01:53.921986 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97bd14-8903-49ea-9058-51fdfe5567fd admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:01:53.923416 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-ea97bd14-8903-49ea-9058-51fdfe5567fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:01:53.980494 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97bd14-8903-49ea-9058-51fdfe5567fd admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:01:53.984217 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673fed1f-9a43-42dd-a31a-b426055e95dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:01:53.984824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fed1f-9a43-42dd-a31a-b426055e95dd service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:53.985121 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fed1f-9a43-42dd-a31a-b426055e95dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:53.985376 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fed1f-9a43-42dd-a31a-b426055e95dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:53.995015 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673fed1f-9a43-42dd-a31a-b426055e95dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:01:53.997391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673fed1f-9a43-42dd-a31a-b426055e95dd service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:01:54.007005 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1396/2798] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:01:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:01:54.271630 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c36043-a4dd-4cd8-9ddd-17f265ced284 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:01:54.273061 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e7c36043-a4dd-4cd8-9ddd-17f265ced284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:01:54.287546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c36043-a4dd-4cd8-9ddd-17f265ced284 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:01:54.292820 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe7939-e546-49f2-91ac-0710276cb272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:01:54.293608 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe7939-e546-49f2-91ac-0710276cb272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:54.293884 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe7939-e546-49f2-91ac-0710276cb272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:54.294149 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe7939-e546-49f2-91ac-0710276cb272 admin admin] PATH_INFO: `/v3/projects/0a4fbdb025bd4ceb9b47f7a91c998862` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:54.295902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abe7939-e546-49f2-91ac-0710276cb272 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a4fbdb025bd4ceb9b47f7a91c998862)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:01:54.298085 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abe7939-e546-49f2-91ac-0710276cb272 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:01:54.345762 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-4abe7939-e546-49f2-91ac-0710276cb272 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a4fbdb025bd4ceb9b47f7a91c998862', 'request_id': 'req-4abe7939-e546-49f2-91ac-0710276cb272'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:01:54.351066 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-4abe7939-e546-49f2-91ac-0710276cb272 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a4fbdb025bd4ceb9b47f7a91c998862', 'request_id': 'req-4abe7939-e546-49f2-91ac-0710276cb272'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:01:54.357773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-4abe7939-e546-49f2-91ac-0710276cb272 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a4fbdb025bd4ceb9b47f7a91c998862', 'request_id': 'req-4abe7939-e546-49f2-91ac-0710276cb272'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:01:54.371295 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-4abe7939-e546-49f2-91ac-0710276cb272 admin admin] The token cache is being invalidate because project 0a4fbdb025bd4ceb9b47f7a91c998862 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:01:54.371583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-4abe7939-e546-49f2-91ac-0710276cb272 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:01:54.372215 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-4abe7939-e546-49f2-91ac-0710276cb272 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:01:54.373686 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1403/2799] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 01:01:54 2022] DELETE /identity/v3/projects/0a4fbdb025bd4ceb9b47f7a91c998862 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:01:54.378687 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140bebd-0f5f-4147-a104-9f473d502ade None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:54.379054 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140bebd-0f5f-4147-a104-9f473d502ade None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:54.379333 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140bebd-0f5f-4147-a104-9f473d502ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:54.399158 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-e140bebd-0f5f-4147-a104-9f473d502ade None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:01:54.400377 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-e140bebd-0f5f-4147-a104-9f473d502ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:01:54.434606 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:01:54.462000 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1397/2800] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:01:54.471381 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26219e2-83b0-4773-88b3-4466840339d6 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:54.471745 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26219e2-83b0-4773-88b3-4466840339d6 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:54.472093 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26219e2-83b0-4773-88b3-4466840339d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:54.492381 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-b26219e2-83b0-4773-88b3-4466840339d6 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:01:54.493576 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b26219e2-83b0-4773-88b3-4466840339d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:01:54.502359 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1404/2801] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:01:54.506603 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140bebd-0f5f-4147-a104-9f473d502ade None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:01:54.516350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140bebd-0f5f-4147-a104-9f473d502ade None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:01:54.520939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb58e60-b831-4b69-b573-5bac6468e276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:01:54.521577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb58e60-b831-4b69-b573-5bac6468e276 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:54.521885 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb58e60-b831-4b69-b573-5bac6468e276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:54.522155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb58e60-b831-4b69-b573-5bac6468e276 admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:54.522787 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb58e60-b831-4b69-b573-5bac6468e276 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:01:54.523124 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb58e60-b831-4b69-b573-5bac6468e276 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:01:54.524731 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb58e60-b831-4b69-b573-5bac6468e276 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:01:54.536270 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1398/2802] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 01:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:01:54.540587 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26219e2-83b0-4773-88b3-4466840339d6 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:01:54.550274 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26219e2-83b0-4773-88b3-4466840339d6 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:01:54.554887 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ac027-3760-44cb-95cc-a04523380160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:01:54.555510 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ac027-3760-44cb-95cc-a04523380160 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:54.555785 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ac027-3760-44cb-95cc-a04523380160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:54.556051 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ac027-3760-44cb-95cc-a04523380160 admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:54.556673 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ac027-3760-44cb-95cc-a04523380160 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:01:54.559376 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ac027-3760-44cb-95cc-a04523380160 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:01:54.585955 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1405/2803] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 01:01:54 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:01:54.590214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb58e60-b831-4b69-b573-5bac6468e276 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:01:54.600578 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb58e60-b831-4b69-b573-5bac6468e276 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:01:54.656420 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ddb89-51c3-489e-adea-82f9d3a97c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:01:54.656937 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ddb89-51c3-489e-adea-82f9d3a97c5c admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:54.657213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ddb89-51c3-489e-adea-82f9d3a97c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:54.657483 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ddb89-51c3-489e-adea-82f9d3a97c5c admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:54.658142 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ddb89-51c3-489e-adea-82f9d3a97c5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:01:54.660073 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ddb89-51c3-489e-adea-82f9d3a97c5c admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:01:54.682021 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1399/2804] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 01:01:54 2022] POST /identity/v3/users => generated 283 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:01:54.686898 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ac027-3760-44cb-95cc-a04523380160 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:01:54.697257 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ac027-3760-44cb-95cc-a04523380160 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:01:54.702691 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d69474-34fb-4a81-b563-22fc5b24c450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:01:54.703331 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d69474-34fb-4a81-b563-22fc5b24c450 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:54.703614 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d69474-34fb-4a81-b563-22fc5b24c450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:54.703890 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d69474-34fb-4a81-b563-22fc5b24c450 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:54.704562 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d69474-34fb-4a81-b563-22fc5b24c450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:01:54.706149 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d69474-34fb-4a81-b563-22fc5b24c450 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:01:54.716113 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1406/2805] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:01:54.720420 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ddb89-51c3-489e-adea-82f9d3a97c5c admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:01:54.731148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ddb89-51c3-489e-adea-82f9d3a97c5c admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:01:54.735850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca730d-5983-4464-9e28-aafb4e2bde69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:01:54.736628 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca730d-5983-4464-9e28-aafb4e2bde69 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:54.736814 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca730d-5983-4464-9e28-aafb4e2bde69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:54.737122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca730d-5983-4464-9e28-aafb4e2bde69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:54.737761 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca730d-5983-4464-9e28-aafb4e2bde69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:01:54.739393 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca730d-5983-4464-9e28-aafb4e2bde69 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:01:54.751474 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1400/2806] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 01:01:54.755746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d69474-34fb-4a81-b563-22fc5b24c450 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:01:54.766924 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d69474-34fb-4a81-b563-22fc5b24c450 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:01:54.771701 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efd3f7-ae02-4f53-9316-412536e39160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:01:54.772352 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efd3f7-ae02-4f53-9316-412536e39160 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:54.772638 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efd3f7-ae02-4f53-9316-412536e39160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:54.772914 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efd3f7-ae02-4f53-9316-412536e39160 admin admin] PATH_INFO: `/v3/projects/630542ad69994af5ab5d499617e39ca1/users/ba934280d4f94f5f8ddff98fdc9c7807/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:54.775251 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:01:54.790616 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6efd3f7-ae02-4f53-9316-412536e39160 admin admin] RBAC: Authorizing `identity:create_grant(project_id=630542ad69994af5ab5d499617e39ca1, user_id=ba934280d4f94f5f8ddff98fdc9c7807, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:01:54.794229 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6efd3f7-ae02-4f53-9316-412536e39160 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:01:54.806845 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1407/2807] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 01:01:54 2022] PUT /identity/v3/projects/630542ad69994af5ab5d499617e39ca1/users/ba934280d4f94f5f8ddff98fdc9c7807/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:01:54.817719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c38da0-1a6e-40fe-83eb-78ba061c1e56 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:54.818016 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c38da0-1a6e-40fe-83eb-78ba061c1e56 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:54.818311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c38da0-1a6e-40fe-83eb-78ba061c1e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:54.846252 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-07c38da0-1a6e-40fe-83eb-78ba061c1e56 None None] MFA Rules not processed for user `ba934280d4f94f5f8ddff98fdc9c7807`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:01:54.847524 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-07c38da0-1a6e-40fe-83eb-78ba061c1e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:01:54.911027 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1401/2808] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:01:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:01:54.922871 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efd3f7-ae02-4f53-9316-412536e39160 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:01:54.924619 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-c6efd3f7-ae02-4f53-9316-412536e39160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:01:54.982165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efd3f7-ae02-4f53-9316-412536e39160 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:01:54.986107 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926cbae-c4ca-48ad-9934-044728b597d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:01:54.986716 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926cbae-c4ca-48ad-9934-044728b597d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:01:54.987028 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926cbae-c4ca-48ad-9934-044728b597d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:01:54.987314 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926cbae-c4ca-48ad-9934-044728b597d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:01:54.996472 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926cbae-c4ca-48ad-9934-044728b597d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:01:54.997910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926cbae-c4ca-48ad-9934-044728b597d4 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:01:55.006995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1408/2809] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:01:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:02:04.907860 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:02:05.002922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:02:20.696101 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c38da0-1a6e-40fe-83eb-78ba061c1e56 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:02:20.697514 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-07c38da0-1a6e-40fe-83eb-78ba061c1e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:02:20.772137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c38da0-1a6e-40fe-83eb-78ba061c1e56 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:02:20.776064 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a557e-5c63-4b06-baa8-0a14312abbf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:02:20.776671 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a557e-5c63-4b06-baa8-0a14312abbf2 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:02:20.776949 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a557e-5c63-4b06-baa8-0a14312abbf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:02:20.777223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a557e-5c63-4b06-baa8-0a14312abbf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:02:20.779769 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-d13a557e-5c63-4b06-baa8-0a14312abbf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:02:20.788979 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13a557e-5c63-4b06-baa8-0a14312abbf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:02:20.790408 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13a557e-5c63-4b06-baa8-0a14312abbf2 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:02:20.814109 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1402/2810] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:02:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:02:30.798216 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=32 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:02:32.838370 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926cbae-c4ca-48ad-9934-044728b597d4 service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:02:32.839993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e926cbae-c4ca-48ad-9934-044728b597d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:02:32.855926 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926cbae-c4ca-48ad-9934-044728b597d4 service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:02:32.860087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241c878-a571-4e08-9451-f380732798e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:02:32.860695 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241c878-a571-4e08-9451-f380732798e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:02:32.860975 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241c878-a571-4e08-9451-f380732798e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:02:32.861245 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241c878-a571-4e08-9451-f380732798e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:02:32.863398 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-5241c878-a571-4e08-9451-f380732798e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:02:32.872982 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5241c878-a571-4e08-9451-f380732798e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:02:32.874430 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5241c878-a571-4e08-9451-f380732798e0 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:02:32.882367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1409/2811] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:02:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:02:33.003554 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a557e-5c63-4b06-baa8-0a14312abbf2 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:02:33.012561 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a557e-5c63-4b06-baa8-0a14312abbf2 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:02:33.017262 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e3d6e-4e9a-4fe7-9de3-f317e73aa326 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:02:33.017988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e3d6e-4e9a-4fe7-9de3-f317e73aa326 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:02:33.018412 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e3d6e-4e9a-4fe7-9de3-f317e73aa326 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:02:33.018740 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e3d6e-4e9a-4fe7-9de3-f317e73aa326 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:02:33.020658 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-8f7e3d6e-4e9a-4fe7-9de3-f317e73aa326 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:02:33.033598 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e3d6e-4e9a-4fe7-9de3-f317e73aa326 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:02:33.035651 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e3d6e-4e9a-4fe7-9de3-f317e73aa326 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:02:33.047381 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1403/2812] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:02:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:02:33.164387 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241c878-a571-4e08-9451-f380732798e0 service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:02:33.173979 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241c878-a571-4e08-9451-f380732798e0 service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:02:33.177651 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924cea68-ec5a-45c8-a071-19e0996ebbaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:02:33.178360 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924cea68-ec5a-45c8-a071-19e0996ebbaa service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:02:33.178640 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924cea68-ec5a-45c8-a071-19e0996ebbaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:02:33.178910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924cea68-ec5a-45c8-a071-19e0996ebbaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:02:33.180692 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-924cea68-ec5a-45c8-a071-19e0996ebbaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:02:33.258821 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924cea68-ec5a-45c8-a071-19e0996ebbaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:02:33.260210 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924cea68-ec5a-45c8-a071-19e0996ebbaa service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:02:33.281675 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1410/2813] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:02:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:02:41.661563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e3d6e-4e9a-4fe7-9de3-f317e73aa326 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:02:41.722737 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e3d6e-4e9a-4fe7-9de3-f317e73aa326 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:02:41.727456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1c5ed-b6d9-4c89-8109-6c95d3051bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:02:41.728092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1c5ed-b6d9-4c89-8109-6c95d3051bde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:02:41.728373 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1c5ed-b6d9-4c89-8109-6c95d3051bde admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:02:41.728643 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1c5ed-b6d9-4c89-8109-6c95d3051bde admin admin] PATH_INFO: `/v3/users/ba934280d4f94f5f8ddff98fdc9c7807` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:02:41.729792 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1c5ed-b6d9-4c89-8109-6c95d3051bde admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba934280d4f94f5f8ddff98fdc9c7807)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:02:41.731744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1c5ed-b6d9-4c89-8109-6c95d3051bde admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:02:41.777223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-7ab1c5ed-b6d9-4c89-8109-6c95d3051bde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba934280d4f94f5f8ddff98fdc9c7807'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:02:41.789127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-7ab1c5ed-b6d9-4c89-8109-6c95d3051bde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba934280d4f94f5f8ddff98fdc9c7807'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:02:41.796529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-7ab1c5ed-b6d9-4c89-8109-6c95d3051bde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba934280d4f94f5f8ddff98fdc9c7807'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:02:41.804092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-7ab1c5ed-b6d9-4c89-8109-6c95d3051bde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba934280d4f94f5f8ddff98fdc9c7807'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:02:41.804813 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-7ab1c5ed-b6d9-4c89-8109-6c95d3051bde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba934280d4f94f5f8ddff98fdc9c7807'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:02:41.810694 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1404/2814] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 01:02:41 2022] DELETE /identity/v3/users/ba934280d4f94f5f8ddff98fdc9c7807 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:02:41.815916 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420484c-1d4b-423a-9053-7cfd927c8e7c None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:02:41.816396 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420484c-1d4b-423a-9053-7cfd927c8e7c None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:02:41.816486 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420484c-1d4b-423a-9053-7cfd927c8e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:02:41.834746 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-6420484c-1d4b-423a-9053-7cfd927c8e7c None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:02:41.835949 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-6420484c-1d4b-423a-9053-7cfd927c8e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:02:41.893774 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1411/2815] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:02:41.903356 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1c5ed-b6d9-4c89-8109-6c95d3051bde admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:02:41.904765 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-7ab1c5ed-b6d9-4c89-8109-6c95d3051bde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:02:41.964214 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1c5ed-b6d9-4c89-8109-6c95d3051bde admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:02:41.968635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a995069-3d42-4373-887e-01deb45960ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:02:41.969762 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a995069-3d42-4373-887e-01deb45960ca service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:02:41.970048 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a995069-3d42-4373-887e-01deb45960ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:02:41.970320 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a995069-3d42-4373-887e-01deb45960ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:02:41.979675 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a995069-3d42-4373-887e-01deb45960ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:02:41.981105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a995069-3d42-4373-887e-01deb45960ca service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:02:41.991706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1405/2816] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:02:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:02:42.257904 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420484c-1d4b-423a-9053-7cfd927c8e7c None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:02:42.259363 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-6420484c-1d4b-423a-9053-7cfd927c8e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:02:42.273474 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420484c-1d4b-423a-9053-7cfd927c8e7c None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:02:42.278246 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79aee7d-609e-46b9-baff-568a204be13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:02:42.278876 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79aee7d-609e-46b9-baff-568a204be13b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:02:42.279193 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79aee7d-609e-46b9-baff-568a204be13b admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:02:42.279465 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79aee7d-609e-46b9-baff-568a204be13b admin admin] PATH_INFO: `/v3/projects/630542ad69994af5ab5d499617e39ca1` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:02:42.280650 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79aee7d-609e-46b9-baff-568a204be13b admin admin] RBAC: Authorizing `identity:delete_project(project_id=630542ad69994af5ab5d499617e39ca1)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:02:42.282720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79aee7d-609e-46b9-baff-568a204be13b admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:02:42.332791 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:02:42.334095 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c79aee7d-609e-46b9-baff-568a204be13b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '630542ad69994af5ab5d499617e39ca1', 'request_id': 'req-c79aee7d-609e-46b9-baff-568a204be13b'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:02:42.340536 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c79aee7d-609e-46b9-baff-568a204be13b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '630542ad69994af5ab5d499617e39ca1', 'request_id': 'req-c79aee7d-609e-46b9-baff-568a204be13b'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:02:42.347097 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c79aee7d-609e-46b9-baff-568a204be13b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '630542ad69994af5ab5d499617e39ca1', 'request_id': 'req-c79aee7d-609e-46b9-baff-568a204be13b'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:02:42.359815 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c79aee7d-609e-46b9-baff-568a204be13b admin admin] The token cache is being invalidate because project 630542ad69994af5ab5d499617e39ca1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:02:42.360206 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c79aee7d-609e-46b9-baff-568a204be13b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:02:42.360902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-c79aee7d-609e-46b9-baff-568a204be13b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:02:42.362474 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1412/2817] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 01:02:42 2022] DELETE /identity/v3/projects/630542ad69994af5ab5d499617e39ca1 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:02:42.367603 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5703b1-7a04-40ab-a87e-8bdd02938aba None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:02:42.367886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5703b1-7a04-40ab-a87e-8bdd02938aba None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:02:42.368156 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5703b1-7a04-40ab-a87e-8bdd02938aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:02:42.387825 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-7a5703b1-7a04-40ab-a87e-8bdd02938aba None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:02:42.389435 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-7a5703b1-7a04-40ab-a87e-8bdd02938aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:02:42.394324 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:02:42.451804 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1406/2818] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:02:42.459807 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55818a61-2036-4817-83c2-c0128a26db5a None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:02:42.460096 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55818a61-2036-4817-83c2-c0128a26db5a None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:02:42.460372 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55818a61-2036-4817-83c2-c0128a26db5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:02:42.480666 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-55818a61-2036-4817-83c2-c0128a26db5a None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:02:42.481930 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-55818a61-2036-4817-83c2-c0128a26db5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:02:42.490662 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1413/2819] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:02:42.494974 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5703b1-7a04-40ab-a87e-8bdd02938aba None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:02:42.505474 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5703b1-7a04-40ab-a87e-8bdd02938aba None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:02:42.510126 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c6529-7b38-46df-983f-639123b07a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:02:42.510729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c6529-7b38-46df-983f-639123b07a8d admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:02:42.511010 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c6529-7b38-46df-983f-639123b07a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:02:42.511344 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c6529-7b38-46df-983f-639123b07a8d admin admin] PATH_INFO: `/v3/domains` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:02:42.512018 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c6529-7b38-46df-983f-639123b07a8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81471) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:02:42.512320 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c6529-7b38-46df-983f-639123b07a8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:02:42.513899 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c6529-7b38-46df-983f-639123b07a8d admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:02:42.528444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1407/2820] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 01:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:02:42.532901 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55818a61-2036-4817-83c2-c0128a26db5a None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:02:42.542193 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55818a61-2036-4817-83c2-c0128a26db5a None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:02:42.547533 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37455f09-8b56-4e7d-bda7-b2a6a5b7b27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:02:42.548137 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37455f09-8b56-4e7d-bda7-b2a6a5b7b27f admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:02:42.548415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37455f09-8b56-4e7d-bda7-b2a6a5b7b27f admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:02:42.548684 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37455f09-8b56-4e7d-bda7-b2a6a5b7b27f admin admin] PATH_INFO: `/v3/projects` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:02:42.549305 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37455f09-8b56-4e7d-bda7-b2a6a5b7b27f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:02:42.551305 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37455f09-8b56-4e7d-bda7-b2a6a5b7b27f admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:02:42.579995 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1414/2821] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 01:02:42 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:02:42.584092 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c6529-7b38-46df-983f-639123b07a8d admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:02:42.595253 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c6529-7b38-46df-983f-639123b07a8d admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:02:42.658698 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3346d5-c88c-4a22-9543-2eb1f0c94337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:02:42.659258 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3346d5-c88c-4a22-9543-2eb1f0c94337 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:02:42.659538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3346d5-c88c-4a22-9543-2eb1f0c94337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:02:42.659810 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3346d5-c88c-4a22-9543-2eb1f0c94337 admin admin] PATH_INFO: `/v3/users` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:02:42.660452 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3346d5-c88c-4a22-9543-2eb1f0c94337 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:02:42.662388 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3346d5-c88c-4a22-9543-2eb1f0c94337 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:02:42.683964 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1408/2822] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 01:02:42 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:02:42.689248 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37455f09-8b56-4e7d-bda7-b2a6a5b7b27f admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:02:42.699777 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37455f09-8b56-4e7d-bda7-b2a6a5b7b27f admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:02:42.705207 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a4a79-9774-4e95-b2c0-940ef54ad8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:02:42.705805 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a4a79-9774-4e95-b2c0-940ef54ad8c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:02:42.706083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a4a79-9774-4e95-b2c0-940ef54ad8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:02:42.706356 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a4a79-9774-4e95-b2c0-940ef54ad8c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:02:42.707004 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3a4a79-9774-4e95-b2c0-940ef54ad8c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:02:42.708627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3a4a79-9774-4e95-b2c0-940ef54ad8c7 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:02:42.718362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1415/2823] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:02:42.722639 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3346d5-c88c-4a22-9543-2eb1f0c94337 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:02:42.732543 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3346d5-c88c-4a22-9543-2eb1f0c94337 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:02:42.737173 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded54898-f98c-4772-8004-437de24e59f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:02:42.737764 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded54898-f98c-4772-8004-437de24e59f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:02:42.738043 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded54898-f98c-4772-8004-437de24e59f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:02:42.738316 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded54898-f98c-4772-8004-437de24e59f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:02:42.738952 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded54898-f98c-4772-8004-437de24e59f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:02:42.740573 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded54898-f98c-4772-8004-437de24e59f8 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:02:42.751442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1409/2824] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:02:42.755627 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a4a79-9774-4e95-b2c0-940ef54ad8c7 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:02:42.766326 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a4a79-9774-4e95-b2c0-940ef54ad8c7 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:02:42.771323 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369af00-9ba1-4fc1-804f-20beb36854ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:02:42.771968 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369af00-9ba1-4fc1-804f-20beb36854ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:02:42.772250 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369af00-9ba1-4fc1-804f-20beb36854ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:02:42.772521 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369af00-9ba1-4fc1-804f-20beb36854ec admin admin] PATH_INFO: `/v3/projects/2a998579761145feaa91931ff92d16ef/users/ae963a2068ac476393110be5f31ce4c4/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:02:42.787395 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7369af00-9ba1-4fc1-804f-20beb36854ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a998579761145feaa91931ff92d16ef, user_id=ae963a2068ac476393110be5f31ce4c4, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:02:42.790038 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7369af00-9ba1-4fc1-804f-20beb36854ec admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:02:42.803172 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1416/2825] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 01:02:42 2022] PUT /identity/v3/projects/2a998579761145feaa91931ff92d16ef/users/ae963a2068ac476393110be5f31ce4c4/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:02:42.814102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998190e-efa7-41ec-99ba-7a52cdd54f5c None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:02:42.814355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998190e-efa7-41ec-99ba-7a52cdd54f5c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:02:42.814590 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998190e-efa7-41ec-99ba-7a52cdd54f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:02:42.842007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-a998190e-efa7-41ec-99ba-7a52cdd54f5c None None] MFA Rules not processed for user `ae963a2068ac476393110be5f31ce4c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:02:42.843256 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-a998190e-efa7-41ec-99ba-7a52cdd54f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:02:42.907577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1410/2826] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:02:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:02:42.918434 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369af00-9ba1-4fc1-804f-20beb36854ec admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:02:42.920194 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-7369af00-9ba1-4fc1-804f-20beb36854ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:02:42.977656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369af00-9ba1-4fc1-804f-20beb36854ec admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:02:42.981562 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81bc44-c20e-4c3b-ae5f-262d903bd35a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:02:42.982170 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81bc44-c20e-4c3b-ae5f-262d903bd35a service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:02:42.982454 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81bc44-c20e-4c3b-ae5f-262d903bd35a service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:02:42.982728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81bc44-c20e-4c3b-ae5f-262d903bd35a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:02:42.991558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81bc44-c20e-4c3b-ae5f-262d903bd35a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:02:42.992990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c81bc44-c20e-4c3b-ae5f-262d903bd35a service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:02:43.002989 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1417/2827] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:02:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:02:44.360016 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9435ff9-c8e5-40b3-9844-1f5e5e6e392a None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:02:44.360566 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9435ff9-c8e5-40b3-9844-1f5e5e6e392a None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:02:44.360566 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9435ff9-c8e5-40b3-9844-1f5e5e6e392a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:02:44.368701 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:02:44.383984 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-c9435ff9-c8e5-40b3-9844-1f5e5e6e392a None None] MFA Rules not processed for user `ae963a2068ac476393110be5f31ce4c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:02:44.385201 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c9435ff9-c8e5-40b3-9844-1f5e5e6e392a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:02:44.393185 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1411/2828] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:02:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:02:52.997793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=60,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:02:54.378927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:02.769158 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6444bad-2e5c-427b-8351-909d27447421 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:02.769667 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6444bad-2e5c-427b-8351-909d27447421 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:02.769817 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6444bad-2e5c-427b-8351-909d27447421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:02.789426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-e6444bad-2e5c-427b-8351-909d27447421 None None] MFA Rules not processed for user `a8fb7a9acefd4b418c992661037c2ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:02.790632 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e6444bad-2e5c-427b-8351-909d27447421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:02.846511 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1418/2829] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 31 01:03:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:03:02.893304 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9435ff9-c8e5-40b3-9844-1f5e5e6e392a None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:02.894737 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c9435ff9-c8e5-40b3-9844-1f5e5e6e392a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:02.962516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9435ff9-c8e5-40b3-9844-1f5e5e6e392a None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:02.967568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08812960-9209-435c-9cf6-21c2c65a3ef1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd3cfbc8ebd4c5a83bef09efe10bc91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:02.968745 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08812960-9209-435c-9cf6-21c2c65a3ef1 service swift] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:02.969107 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08812960-9209-435c-9cf6-21c2c65a3ef1 service swift] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:02.969639 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08812960-9209-435c-9cf6-21c2c65a3ef1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:02.978080 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08812960-9209-435c-9cf6-21c2c65a3ef1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:02.979704 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08812960-9209-435c-9cf6-21c2c65a3ef1 service swift] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:02.987963 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1412/2830] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 31 01:03:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 01:03:03.490503 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21905329-ae61-4ee2-a282-d59ec684603a None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:03.490792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21905329-ae61-4ee2-a282-d59ec684603a None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:03.491096 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21905329-ae61-4ee2-a282-d59ec684603a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:03.511147 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-21905329-ae61-4ee2-a282-d59ec684603a None None] MFA Rules not processed for user `a8fb7a9acefd4b418c992661037c2ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:03.512338 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-21905329-ae61-4ee2-a282-d59ec684603a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:03.519558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1419/2831] 10.222.0.67 () {58 vars in 967 bytes} [Wed Aug 31 01:03:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:03:03.531794 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08812960-9209-435c-9cf6-21c2c65a3ef1 service swift] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:03.542652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08812960-9209-435c-9cf6-21c2c65a3ef1 service swift] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:03.548215 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7c6ac-fdd4-440d-ac80-201640a827f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd3cfbc8ebd4c5a83bef09efe10bc91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:03.548810 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7c6ac-fdd4-440d-ac80-201640a827f9 service swift] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:03.549088 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7c6ac-fdd4-440d-ac80-201640a827f9 service swift] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:03.549355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7c6ac-fdd4-440d-ac80-201640a827f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:03.557460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca7c6ac-fdd4-440d-ac80-201640a827f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:03.559023 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca7c6ac-fdd4-440d-ac80-201640a827f9 service swift] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:03.567552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1413/2832] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 31 01:03:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 01:03:03.807347 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849120c3-5d0d-4292-a2cc-c9615f8d8749 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:03.807864 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849120c3-5d0d-4292-a2cc-c9615f8d8749 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:03.807955 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849120c3-5d0d-4292-a2cc-c9615f8d8749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:03.828501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-849120c3-5d0d-4292-a2cc-c9615f8d8749 None None] MFA Rules not processed for user `a8fb7a9acefd4b418c992661037c2ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:03.829684 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-849120c3-5d0d-4292-a2cc-c9615f8d8749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:03.836771 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1420/2833] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 31 01:03:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:03:03.849567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7c6ac-fdd4-440d-ac80-201640a827f9 service swift] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:03.860566 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7c6ac-fdd4-440d-ac80-201640a827f9 service swift] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:03.865518 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f81055-3fd4-4c06-bc9c-f1292f8369c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd3cfbc8ebd4c5a83bef09efe10bc91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:03.866106 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f81055-3fd4-4c06-bc9c-f1292f8369c6 service swift] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:03.866387 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f81055-3fd4-4c06-bc9c-f1292f8369c6 service swift] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:03.866667 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f81055-3fd4-4c06-bc9c-f1292f8369c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:03.875110 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f81055-3fd4-4c06-bc9c-f1292f8369c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:03.876676 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f81055-3fd4-4c06-bc9c-f1292f8369c6 service swift] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:03.886538 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1414/2834] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 31 01:03:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 01:03:04.221041 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163cb42-66a7-49c8-a6f9-40d9519cf081 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:04.221331 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163cb42-66a7-49c8-a6f9-40d9519cf081 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:04.221601 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163cb42-66a7-49c8-a6f9-40d9519cf081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:04.240836 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-9163cb42-66a7-49c8-a6f9-40d9519cf081 None None] MFA Rules not processed for user `a8fb7a9acefd4b418c992661037c2ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:04.242035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-9163cb42-66a7-49c8-a6f9-40d9519cf081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:04.249676 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1421/2835] 10.222.0.67 () {58 vars in 967 bytes} [Wed Aug 31 01:03:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:03:04.261585 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f81055-3fd4-4c06-bc9c-f1292f8369c6 service swift] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:04.272953 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f81055-3fd4-4c06-bc9c-f1292f8369c6 service swift] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:04.277908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e4d2c6-2ba6-4223-b78f-23cc62cd0f3c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd3cfbc8ebd4c5a83bef09efe10bc91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:04.278497 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4d2c6-2ba6-4223-b78f-23cc62cd0f3c service swift] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:04.278774 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4d2c6-2ba6-4223-b78f-23cc62cd0f3c service swift] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:04.279055 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e4d2c6-2ba6-4223-b78f-23cc62cd0f3c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:04.288395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e4d2c6-2ba6-4223-b78f-23cc62cd0f3c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:04.289961 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e4d2c6-2ba6-4223-b78f-23cc62cd0f3c service swift] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:04.298495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1415/2836] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 31 01:03:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 01:03:06.489096 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163cb42-66a7-49c8-a6f9-40d9519cf081 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:06.556249 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163cb42-66a7-49c8-a6f9-40d9519cf081 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:06.560769 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9a55d-adf1-4baf-8d44-b53a7762a882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:06.561543 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9a55d-adf1-4baf-8d44-b53a7762a882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:06.561835 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9a55d-adf1-4baf-8d44-b53a7762a882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:06.562222 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9a55d-adf1-4baf-8d44-b53a7762a882 admin admin] PATH_INFO: `/v3/users/ae963a2068ac476393110be5f31ce4c4` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:06.563569 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9a55d-adf1-4baf-8d44-b53a7762a882 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae963a2068ac476393110be5f31ce4c4)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:06.565310 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9a55d-adf1-4baf-8d44-b53a7762a882 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:06.615126 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-cdd9a55d-adf1-4baf-8d44-b53a7762a882 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae963a2068ac476393110be5f31ce4c4'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:03:06.624039 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-cdd9a55d-adf1-4baf-8d44-b53a7762a882 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae963a2068ac476393110be5f31ce4c4'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:03:06.631584 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-cdd9a55d-adf1-4baf-8d44-b53a7762a882 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae963a2068ac476393110be5f31ce4c4'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:03:06.632315 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-cdd9a55d-adf1-4baf-8d44-b53a7762a882 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae963a2068ac476393110be5f31ce4c4'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:03:06.632925 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-cdd9a55d-adf1-4baf-8d44-b53a7762a882 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae963a2068ac476393110be5f31ce4c4'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:03:06.652057 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1422/2837] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 01:03:06 2022] DELETE /identity/v3/users/ae963a2068ac476393110be5f31ce4c4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:03:06.656767 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab7fa1-9e42-4719-b3f9-788926c0aba5 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:06.657274 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab7fa1-9e42-4719-b3f9-788926c0aba5 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:06.657376 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab7fa1-9e42-4719-b3f9-788926c0aba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:06.676768 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-13ab7fa1-9e42-4719-b3f9-788926c0aba5 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:06.678064 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-13ab7fa1-9e42-4719-b3f9-788926c0aba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:06.739137 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1416/2838] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:03:06.749506 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9a55d-adf1-4baf-8d44-b53a7762a882 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:06.751436 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-cdd9a55d-adf1-4baf-8d44-b53a7762a882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:06.758200 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:06.813049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9a55d-adf1-4baf-8d44-b53a7762a882 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:06.816675 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a503ce4-6816-430f-9b6d-a6b1322fbbd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:06.817393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a503ce4-6816-430f-9b6d-a6b1322fbbd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:06.817688 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a503ce4-6816-430f-9b6d-a6b1322fbbd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:06.817932 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a503ce4-6816-430f-9b6d-a6b1322fbbd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:06.827023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a503ce4-6816-430f-9b6d-a6b1322fbbd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:06.828573 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a503ce4-6816-430f-9b6d-a6b1322fbbd0 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:06.838154 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1423/2839] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:03:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:03:07.011897 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab7fa1-9e42-4719-b3f9-788926c0aba5 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:07.013961 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-13ab7fa1-9e42-4719-b3f9-788926c0aba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:07.026612 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab7fa1-9e42-4719-b3f9-788926c0aba5 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:07.032460 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6656e648-1ad0-4041-a490-b84109681dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:07.033296 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6656e648-1ad0-4041-a490-b84109681dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:07.033744 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6656e648-1ad0-4041-a490-b84109681dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:07.034635 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6656e648-1ad0-4041-a490-b84109681dcd admin admin] PATH_INFO: `/v3/projects/2a998579761145feaa91931ff92d16ef` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:07.035233 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6656e648-1ad0-4041-a490-b84109681dcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a998579761145feaa91931ff92d16ef)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:07.037158 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6656e648-1ad0-4041-a490-b84109681dcd admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:07.083202 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:07.086385 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-6656e648-1ad0-4041-a490-b84109681dcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a998579761145feaa91931ff92d16ef', 'request_id': 'req-6656e648-1ad0-4041-a490-b84109681dcd'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:03:07.094865 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-6656e648-1ad0-4041-a490-b84109681dcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a998579761145feaa91931ff92d16ef', 'request_id': 'req-6656e648-1ad0-4041-a490-b84109681dcd'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:03:07.101622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-6656e648-1ad0-4041-a490-b84109681dcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a998579761145feaa91931ff92d16ef', 'request_id': 'req-6656e648-1ad0-4041-a490-b84109681dcd'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:03:07.115683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-6656e648-1ad0-4041-a490-b84109681dcd admin admin] The token cache is being invalidate because project 2a998579761145feaa91931ff92d16ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:03:07.115954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-6656e648-1ad0-4041-a490-b84109681dcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:03:07.116808 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-6656e648-1ad0-4041-a490-b84109681dcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:03:07.118434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1417/2840] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 01:03:07 2022] DELETE /identity/v3/projects/2a998579761145feaa91931ff92d16ef => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:03:07.124608 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf2264-6328-47c5-a652-477488986f59 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:07.125025 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf2264-6328-47c5-a652-477488986f59 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:07.125311 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf2264-6328-47c5-a652-477488986f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:07.147366 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-babf2264-6328-47c5-a652-477488986f59 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:07.148693 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-babf2264-6328-47c5-a652-477488986f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:07.213021 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1424/2841] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:03:07.222102 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba589d-225b-4ee9-b2a7-8560731d16db None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:07.222388 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba589d-225b-4ee9-b2a7-8560731d16db None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:07.222693 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba589d-225b-4ee9-b2a7-8560731d16db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:07.243962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-57ba589d-225b-4ee9-b2a7-8560731d16db None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:07.245179 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-57ba589d-225b-4ee9-b2a7-8560731d16db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:07.253459 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1418/2842] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:03:07.257814 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf2264-6328-47c5-a652-477488986f59 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:07.268869 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf2264-6328-47c5-a652-477488986f59 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:07.273633 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9661fad-5161-4f2f-a82f-42669b5a6de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:07.274314 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9661fad-5161-4f2f-a82f-42669b5a6de9 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:07.274667 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9661fad-5161-4f2f-a82f-42669b5a6de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:07.275011 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9661fad-5161-4f2f-a82f-42669b5a6de9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:07.275751 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9661fad-5161-4f2f-a82f-42669b5a6de9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:03:07.276054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9661fad-5161-4f2f-a82f-42669b5a6de9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:07.277682 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9661fad-5161-4f2f-a82f-42669b5a6de9 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:07.290091 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1425/2843] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 01:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:03:07.294526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba589d-225b-4ee9-b2a7-8560731d16db None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:07.305220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba589d-225b-4ee9-b2a7-8560731d16db None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:07.311718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330653e9-1a0e-4d99-8ae3-7caeee6db39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:07.312327 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330653e9-1a0e-4d99-8ae3-7caeee6db39e admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:07.312605 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330653e9-1a0e-4d99-8ae3-7caeee6db39e admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:07.312926 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330653e9-1a0e-4d99-8ae3-7caeee6db39e admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:07.318828 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330653e9-1a0e-4d99-8ae3-7caeee6db39e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:07.320787 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330653e9-1a0e-4d99-8ae3-7caeee6db39e admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:07.359230 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1419/2844] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 01:03:07 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:03:07.362164 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9661fad-5161-4f2f-a82f-42669b5a6de9 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:07.372766 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9661fad-5161-4f2f-a82f-42669b5a6de9 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:07.435276 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb11ffb-62ef-4d6b-9701-3548b881b19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:07.435905 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb11ffb-62ef-4d6b-9701-3548b881b19e admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:07.436216 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb11ffb-62ef-4d6b-9701-3548b881b19e admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:07.437035 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb11ffb-62ef-4d6b-9701-3548b881b19e admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:07.437785 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb11ffb-62ef-4d6b-9701-3548b881b19e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:07.439744 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb11ffb-62ef-4d6b-9701-3548b881b19e admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:07.461853 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1426/2845] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 01:03:07 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:03:07.466159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330653e9-1a0e-4d99-8ae3-7caeee6db39e admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:07.476825 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330653e9-1a0e-4d99-8ae3-7caeee6db39e admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:07.481514 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d0b0c-6b2c-41ce-a9aa-464370dcfcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:07.482120 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d0b0c-6b2c-41ce-a9aa-464370dcfcaf admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:07.482400 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d0b0c-6b2c-41ce-a9aa-464370dcfcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:07.482670 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d0b0c-6b2c-41ce-a9aa-464370dcfcaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:07.483910 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d0b0c-6b2c-41ce-a9aa-464370dcfcaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:07.485588 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d0b0c-6b2c-41ce-a9aa-464370dcfcaf admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:07.495893 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1420/2846] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:03:07.500801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb11ffb-62ef-4d6b-9701-3548b881b19e admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:07.511261 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb11ffb-62ef-4d6b-9701-3548b881b19e admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:07.516169 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7975e1a-179f-4c48-bb14-025740a6edc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:07.516784 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7975e1a-179f-4c48-bb14-025740a6edc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:07.517060 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7975e1a-179f-4c48-bb14-025740a6edc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:07.517329 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7975e1a-179f-4c48-bb14-025740a6edc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:07.517975 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7975e1a-179f-4c48-bb14-025740a6edc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:07.519630 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7975e1a-179f-4c48-bb14-025740a6edc7 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:07.531263 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1427/2847] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:03:07.535438 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d0b0c-6b2c-41ce-a9aa-464370dcfcaf admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:07.546692 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d0b0c-6b2c-41ce-a9aa-464370dcfcaf admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:07.552105 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b333c-1db0-4b11-a748-4739ffd4072e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:07.552794 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b333c-1db0-4b11-a748-4739ffd4072e admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:07.553078 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b333c-1db0-4b11-a748-4739ffd4072e admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:07.553353 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b333c-1db0-4b11-a748-4739ffd4072e admin admin] PATH_INFO: `/v3/projects/7f85c7d484fa4aebafc3c0a12c9569c2/users/74ec301c62e541e0851f7570bcf07e4f/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:07.570230 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b333c-1db0-4b11-a748-4739ffd4072e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f85c7d484fa4aebafc3c0a12c9569c2, user_id=74ec301c62e541e0851f7570bcf07e4f, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:07.573002 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b333c-1db0-4b11-a748-4739ffd4072e admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:07.587391 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1421/2848] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 01:03:07 2022] PUT /identity/v3/projects/7f85c7d484fa4aebafc3c0a12c9569c2/users/74ec301c62e541e0851f7570bcf07e4f/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:03:07.593029 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f18883-4eb0-449b-8515-977e25ae9ed7 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:07.593455 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f18883-4eb0-449b-8515-977e25ae9ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:07.593795 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f18883-4eb0-449b-8515-977e25ae9ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:07.614833 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-67f18883-4eb0-449b-8515-977e25ae9ed7 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:07.616171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-67f18883-4eb0-449b-8515-977e25ae9ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:07.626743 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:07.686839 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1428/2849] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:03:07.696709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b333c-1db0-4b11-a748-4739ffd4072e admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:07.698749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-1a9b333c-1db0-4b11-a748-4739ffd4072e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:07.777689 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b333c-1db0-4b11-a748-4739ffd4072e admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:07.781828 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167921c-1004-4147-a559-9dff582f7372 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:07.782439 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167921c-1004-4147-a559-9dff582f7372 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:07.782717 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167921c-1004-4147-a559-9dff582f7372 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:07.782984 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167921c-1004-4147-a559-9dff582f7372 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:07.793265 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167921c-1004-4147-a559-9dff582f7372 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:07.794774 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167921c-1004-4147-a559-9dff582f7372 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:07.804894 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1422/2850] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:03:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:03:12.071731 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb04b9-0667-4173-81b4-5a2f8253d633 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:12.072274 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb04b9-0667-4173-81b4-5a2f8253d633 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:12.072490 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb04b9-0667-4173-81b4-5a2f8253d633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:12.104566 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-46fb04b9-0667-4173-81b4-5a2f8253d633 None None] MFA Rules not processed for user `74ec301c62e541e0851f7570bcf07e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:12.106589 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-46fb04b9-0667-4173-81b4-5a2f8253d633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:12.201074 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1429/2851] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:03:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:03:12.206482 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167921c-1004-4147-a559-9dff582f7372 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:12.222632 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167921c-1004-4147-a559-9dff582f7372 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:12.228268 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f857218-d8e2-4373-9581-3ce991a6614d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:12.229447 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f857218-d8e2-4373-9581-3ce991a6614d admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:12.229879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f857218-d8e2-4373-9581-3ce991a6614d admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:12.230298 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f857218-d8e2-4373-9581-3ce991a6614d admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:12.231148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f857218-d8e2-4373-9581-3ce991a6614d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:12.234726 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f857218-d8e2-4373-9581-3ce991a6614d admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:12.270582 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1423/2852] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 01:03:12 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:03:12.274652 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb04b9-0667-4173-81b4-5a2f8253d633 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:12.290348 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb04b9-0667-4173-81b4-5a2f8253d633 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:12.364153 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c145e13-8b44-4cee-88e2-19c8944d43a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:12.364772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c145e13-8b44-4cee-88e2-19c8944d43a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:12.365052 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c145e13-8b44-4cee-88e2-19c8944d43a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:12.365323 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c145e13-8b44-4cee-88e2-19c8944d43a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:12.365955 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c145e13-8b44-4cee-88e2-19c8944d43a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:12.368067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c145e13-8b44-4cee-88e2-19c8944d43a7 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:12.405891 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1430/2853] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 01:03:12 2022] POST /identity/v3/users => generated 298 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:03:12.410363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f857218-d8e2-4373-9581-3ce991a6614d admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:12.422139 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f857218-d8e2-4373-9581-3ce991a6614d admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:12.426896 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a5c9b-d91d-4ecd-849a-cd81c0b458ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:12.427556 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a5c9b-d91d-4ecd-849a-cd81c0b458ff admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:12.427992 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a5c9b-d91d-4ecd-849a-cd81c0b458ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:12.428275 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a5c9b-d91d-4ecd-849a-cd81c0b458ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:12.428933 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a5c9b-d91d-4ecd-849a-cd81c0b458ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:12.430790 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a5c9b-d91d-4ecd-849a-cd81c0b458ff admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:12.442169 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1424/2854] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:03:12.446651 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c145e13-8b44-4cee-88e2-19c8944d43a7 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:12.457998 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c145e13-8b44-4cee-88e2-19c8944d43a7 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:12.462976 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5809658-ae13-4265-85b6-523f80442a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:12.463804 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5809658-ae13-4265-85b6-523f80442a7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:12.464160 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5809658-ae13-4265-85b6-523f80442a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:12.464504 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5809658-ae13-4265-85b6-523f80442a7a admin admin] PATH_INFO: `/v3/projects/b21db527ca88436082ea46e41ac5f545/users/4e21a053d6664863a65972e90326935d/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:12.485166 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5809658-ae13-4265-85b6-523f80442a7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b21db527ca88436082ea46e41ac5f545, user_id=4e21a053d6664863a65972e90326935d, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:12.488177 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5809658-ae13-4265-85b6-523f80442a7a admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:12.504917 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1431/2855] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 01:03:12 2022] PUT /identity/v3/projects/b21db527ca88436082ea46e41ac5f545/users/4e21a053d6664863a65972e90326935d/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:03:12.510004 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a5c9b-d91d-4ecd-849a-cd81c0b458ff admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:12.528801 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a5c9b-d91d-4ecd-849a-cd81c0b458ff admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:12.572945 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:12.602164 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b550b2-3eb1-4414-aab9-5ea3bf3e71dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:12.603012 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b550b2-3eb1-4414-aab9-5ea3bf3e71dd admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:12.603399 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b550b2-3eb1-4414-aab9-5ea3bf3e71dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:12.603782 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b550b2-3eb1-4414-aab9-5ea3bf3e71dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:12.604516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b550b2-3eb1-4414-aab9-5ea3bf3e71dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:12.606273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b550b2-3eb1-4414-aab9-5ea3bf3e71dd admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:12.621485 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1425/2856] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:03:12.622862 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5809658-ae13-4265-85b6-523f80442a7a admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:12.634527 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5809658-ae13-4265-85b6-523f80442a7a admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:12.640803 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e61cb3-09dd-4b12-a48e-de2627b360f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:12.641837 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e61cb3-09dd-4b12-a48e-de2627b360f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:12.642121 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e61cb3-09dd-4b12-a48e-de2627b360f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:12.642394 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e61cb3-09dd-4b12-a48e-de2627b360f5 admin admin] PATH_INFO: `/v3/projects/b21db527ca88436082ea46e41ac5f545/users/4e21a053d6664863a65972e90326935d/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:12.644368 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e61cb3-09dd-4b12-a48e-de2627b360f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b21db527ca88436082ea46e41ac5f545, user_id=4e21a053d6664863a65972e90326935d, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:12.647022 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e61cb3-09dd-4b12-a48e-de2627b360f5 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:12.650568 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:12.665440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1432/2857] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 01:03:12 2022] PUT /identity/v3/projects/b21db527ca88436082ea46e41ac5f545/users/4e21a053d6664863a65972e90326935d/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:03:12.846056 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b550b2-3eb1-4414-aab9-5ea3bf3e71dd admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:12.848028 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-64b550b2-3eb1-4414-aab9-5ea3bf3e71dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:13.052875 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b550b2-3eb1-4414-aab9-5ea3bf3e71dd admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:13.200853 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ec572-0022-4794-8ee9-3b472a1b9825 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:13.201477 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ec572-0022-4794-8ee9-3b472a1b9825 service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:13.202302 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ec572-0022-4794-8ee9-3b472a1b9825 service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:13.202625 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ec572-0022-4794-8ee9-3b472a1b9825 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:13.205015 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-413ec572-0022-4794-8ee9-3b472a1b9825 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:13.218219 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ec572-0022-4794-8ee9-3b472a1b9825 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:13.220928 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ec572-0022-4794-8ee9-3b472a1b9825 service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:13.233094 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1426/2858] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:03:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:03:17.710589 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c70d2e-3c05-4663-86ce-35b4241f5d3e None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:17.711170 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c70d2e-3c05-4663-86ce-35b4241f5d3e None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:17.711297 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c70d2e-3c05-4663-86ce-35b4241f5d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:17.751236 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-e1c70d2e-3c05-4663-86ce-35b4241f5d3e None None] MFA Rules not processed for user `26826ddf92124c95835a5356e13040bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:17.752542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e1c70d2e-3c05-4663-86ce-35b4241f5d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:17.836867 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1433/2859] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:03:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:03:18.135353 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95572163-6762-41fc-8eb0-c00d063a4b5c None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:18.136229 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95572163-6762-41fc-8eb0-c00d063a4b5c None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:18.136696 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95572163-6762-41fc-8eb0-c00d063a4b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:18.171659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-95572163-6762-41fc-8eb0-c00d063a4b5c None None] MFA Rules not processed for user `4e21a053d6664863a65972e90326935d`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:18.173282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-95572163-6762-41fc-8eb0-c00d063a4b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:18.233887 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:18.259618 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1427/2860] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:03:18 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:03:18.273830 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c70d2e-3c05-4663-86ce-35b4241f5d3e None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:18.275302 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e1c70d2e-3c05-4663-86ce-35b4241f5d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:18.346998 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c70d2e-3c05-4663-86ce-35b4241f5d3e None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:18.351728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27047538-6e22-4cc2-b4b8-726a198d091e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55714a21c1cb4f2b949603230243ed02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:18.353030 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27047538-6e22-4cc2-b4b8-726a198d091e service glance] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:18.353782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27047538-6e22-4cc2-b4b8-726a198d091e service glance] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:18.354062 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27047538-6e22-4cc2-b4b8-726a198d091e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:18.421653 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27047538-6e22-4cc2-b4b8-726a198d091e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:18.424299 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27047538-6e22-4cc2-b4b8-726a198d091e service glance] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:18.434812 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1434/2861] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:03:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:03:18.591993 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f419f-4959-42f0-99a4-2e8b30d6e5e7 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:18.592415 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f419f-4959-42f0-99a4-2e8b30d6e5e7 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:18.595302 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f419f-4959-42f0-99a4-2e8b30d6e5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:18.616569 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-1f6f419f-4959-42f0-99a4-2e8b30d6e5e7 None None] MFA Rules not processed for user `a8fb7a9acefd4b418c992661037c2ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:18.617871 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-1f6f419f-4959-42f0-99a4-2e8b30d6e5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:18.690874 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1428/2862] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 31 01:03:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:03:18.708841 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27047538-6e22-4cc2-b4b8-726a198d091e service glance] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:18.711384 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-27047538-6e22-4cc2-b4b8-726a198d091e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:18.814807 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=100 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:18.835693 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27047538-6e22-4cc2-b4b8-726a198d091e service glance] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:18.840882 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27007a17-7555-4475-bb9a-ed4ff7eeef15 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd3cfbc8ebd4c5a83bef09efe10bc91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:18.841500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27007a17-7555-4475-bb9a-ed4ff7eeef15 service swift] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:18.841779 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27007a17-7555-4475-bb9a-ed4ff7eeef15 service swift] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:18.842044 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27007a17-7555-4475-bb9a-ed4ff7eeef15 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:18.854484 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27007a17-7555-4475-bb9a-ed4ff7eeef15 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:18.856170 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27007a17-7555-4475-bb9a-ed4ff7eeef15 service swift] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:18.866160 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1435/2863] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 31 01:03:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 01:03:28.401103 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf1b0c-5b15-47e2-bf64-226be6b4457e None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:28.401979 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf1b0c-5b15-47e2-bf64-226be6b4457e None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:28.402418 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf1b0c-5b15-47e2-bf64-226be6b4457e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:28.423275 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-3faf1b0c-5b15-47e2-bf64-226be6b4457e None None] MFA Rules not processed for user `74ec301c62e541e0851f7570bcf07e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:28.424530 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-3faf1b0c-5b15-47e2-bf64-226be6b4457e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:28.431487 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1429/2864] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:03:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:03:28.862380 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=7 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:31.003238 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a975790-f811-41dc-bcf8-41258fc4377c None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:31.003854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a975790-f811-41dc-bcf8-41258fc4377c None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:31.003854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a975790-f811-41dc-bcf8-41258fc4377c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:31.022853 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-9a975790-f811-41dc-bcf8-41258fc4377c None None] MFA Rules not processed for user `a8fb7a9acefd4b418c992661037c2ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:03:31.025576 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-9a975790-f811-41dc-bcf8-41258fc4377c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:31.033953 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1436/2865] 10.222.0.67 () {58 vars in 1009 bytes} [Wed Aug 31 01:03:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:03:31.048888 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf1b0c-5b15-47e2-bf64-226be6b4457e None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:31.062013 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf1b0c-5b15-47e2-bf64-226be6b4457e None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:31.067198 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747adb2-421d-477e-90cc-2b27b02f43c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd3cfbc8ebd4c5a83bef09efe10bc91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:31.067887 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747adb2-421d-477e-90cc-2b27b02f43c5 service swift] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:31.068250 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747adb2-421d-477e-90cc-2b27b02f43c5 service swift] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:31.068720 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747adb2-421d-477e-90cc-2b27b02f43c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:31.078260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747adb2-421d-477e-90cc-2b27b02f43c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:31.080128 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747adb2-421d-477e-90cc-2b27b02f43c5 service swift] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:31.094533 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1430/2866] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 31 01:03:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 01:03:41.017071 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:41.086729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=41 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:03:49.078500 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a975790-f811-41dc-bcf8-41258fc4377c None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:49.170180 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a975790-f811-41dc-bcf8-41258fc4377c None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:49.174848 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b962ec-c7dd-46e5-ae0a-b6c834bc23b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:49.175637 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b962ec-c7dd-46e5-ae0a-b6c834bc23b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:49.175922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b962ec-c7dd-46e5-ae0a-b6c834bc23b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:49.176243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b962ec-c7dd-46e5-ae0a-b6c834bc23b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:49.178315 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-d1b962ec-c7dd-46e5-ae0a-b6c834bc23b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:49.192772 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b962ec-c7dd-46e5-ae0a-b6c834bc23b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:49.194306 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b962ec-c7dd-46e5-ae0a-b6c834bc23b6 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:49.205875 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1437/2867] 10.222.0.67 () {60 vars in 1396 bytes} [Wed Aug 31 01:03:49 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:03:50.188311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747adb2-421d-477e-90cc-2b27b02f43c5 service swift] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:50.200998 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747adb2-421d-477e-90cc-2b27b02f43c5 service swift] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:50.207954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0a049-0411-46ca-abb1-571fb5d5d940 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:50.208564 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0a049-0411-46ca-abb1-571fb5d5d940 service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:50.208846 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0a049-0411-46ca-abb1-571fb5d5d940 service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:50.209123 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0a049-0411-46ca-abb1-571fb5d5d940 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:50.211173 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-2de0a049-0411-46ca-abb1-571fb5d5d940 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:50.224932 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de0a049-0411-46ca-abb1-571fb5d5d940 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:50.226869 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de0a049-0411-46ca-abb1-571fb5d5d940 service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:50.238947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1431/2868] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:03:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:03:55.978546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b962ec-c7dd-46e5-ae0a-b6c834bc23b6 service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:03:55.994650 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b962ec-c7dd-46e5-ae0a-b6c834bc23b6 service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:03:56.001372 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619d087-c6cc-41cf-9191-6e75f637a772 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:03:56.001987 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619d087-c6cc-41cf-9191-6e75f637a772 service nova] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:03:56.002267 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619d087-c6cc-41cf-9191-6e75f637a772 service nova] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:03:56.002543 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619d087-c6cc-41cf-9191-6e75f637a772 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:03:56.004296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-f619d087-c6cc-41cf-9191-6e75f637a772 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:03:56.017434 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619d087-c6cc-41cf-9191-6e75f637a772 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:03:56.019051 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619d087-c6cc-41cf-9191-6e75f637a772 service nova] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:03:56.029778 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1438/2869] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:03:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:04:00.234643 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=6 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:06.025343 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=32 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:08.994044 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0a049-0411-46ca-abb1-571fb5d5d940 service nova] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:09.004451 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0a049-0411-46ca-abb1-571fb5d5d940 service nova] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:09.008061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de626c-1995-4636-a582-05c3a85fe75c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:09.008659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de626c-1995-4636-a582-05c3a85fe75c service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:09.008942 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de626c-1995-4636-a582-05c3a85fe75c service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:09.009217 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de626c-1995-4636-a582-05c3a85fe75c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:09.010938 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-e7de626c-1995-4636-a582-05c3a85fe75c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:09.025223 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de626c-1995-4636-a582-05c3a85fe75c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:09.026809 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de626c-1995-4636-a582-05c3a85fe75c service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:09.038861 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1432/2870] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:04:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:04:09.165706 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619d087-c6cc-41cf-9191-6e75f637a772 service nova] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:09.174991 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619d087-c6cc-41cf-9191-6e75f637a772 service nova] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:09.178514 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90663bbc-aca9-44be-b6d4-37bff32ea081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:09.179138 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90663bbc-aca9-44be-b6d4-37bff32ea081 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:09.179417 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90663bbc-aca9-44be-b6d4-37bff32ea081 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:09.179682 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90663bbc-aca9-44be-b6d4-37bff32ea081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:09.189390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90663bbc-aca9-44be-b6d4-37bff32ea081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:09.190850 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90663bbc-aca9-44be-b6d4-37bff32ea081 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:09.200445 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1439/2871] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:04:09 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:04:12.372140 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de626c-1995-4636-a582-05c3a85fe75c service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:12.382116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de626c-1995-4636-a582-05c3a85fe75c service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:12.385787 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bff614-010a-4545-875f-f66157a66ac6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:12.386389 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bff614-010a-4545-875f-f66157a66ac6 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:12.386725 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bff614-010a-4545-875f-f66157a66ac6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:12.386921 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bff614-010a-4545-875f-f66157a66ac6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:12.388683 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-25bff614-010a-4545-875f-f66157a66ac6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:12.404277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bff614-010a-4545-875f-f66157a66ac6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:12.405899 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bff614-010a-4545-875f-f66157a66ac6 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:12.418030 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1433/2872] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:04:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:04:19.196227 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=6 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:22.412612 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=12 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:38.723871 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90663bbc-aca9-44be-b6d4-37bff32ea081 service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:38.725328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-90663bbc-aca9-44be-b6d4-37bff32ea081 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:38.736564 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90663bbc-aca9-44be-b6d4-37bff32ea081 service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:38.740130 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3fed6e-9708-42ce-8b5e-dff206c4253b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:38.740716 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fed6e-9708-42ce-8b5e-dff206c4253b service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:38.741022 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fed6e-9708-42ce-8b5e-dff206c4253b service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:38.741321 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fed6e-9708-42ce-8b5e-dff206c4253b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:38.742981 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-ff3fed6e-9708-42ce-8b5e-dff206c4253b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:38.801018 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3fed6e-9708-42ce-8b5e-dff206c4253b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:38.802449 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3fed6e-9708-42ce-8b5e-dff206c4253b service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:38.811391 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1440/2873] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:04:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:04:43.753786 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bff614-010a-4545-875f-f66157a66ac6 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:43.762555 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bff614-010a-4545-875f-f66157a66ac6 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:43.766973 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d62313-33ef-44a2-bc31-024d3086b7b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:43.767605 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d62313-33ef-44a2-bc31-024d3086b7b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:43.767886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d62313-33ef-44a2-bc31-024d3086b7b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:43.768158 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d62313-33ef-44a2-bc31-024d3086b7b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:43.769902 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-d2d62313-33ef-44a2-bc31-024d3086b7b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:43.833268 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d62313-33ef-44a2-bc31-024d3086b7b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:43.834704 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d62313-33ef-44a2-bc31-024d3086b7b3 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:43.844550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1434/2874] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:04:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:04:46.670987 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3fed6e-9708-42ce-8b5e-dff206c4253b service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:46.673025 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-ff3fed6e-9708-42ce-8b5e-dff206c4253b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:46.772972 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3fed6e-9708-42ce-8b5e-dff206c4253b service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:46.778022 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ae6b7-cd3f-4618-950e-baeec9ccb0e1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdacdcb2451742938164fdd44cdbaebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:46.778628 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ae6b7-cd3f-4618-950e-baeec9ccb0e1 service placement] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:46.778911 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ae6b7-cd3f-4618-950e-baeec9ccb0e1 service placement] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:46.779222 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ae6b7-cd3f-4618-950e-baeec9ccb0e1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:46.780990 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-639ae6b7-cd3f-4618-950e-baeec9ccb0e1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:46.794405 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ae6b7-cd3f-4618-950e-baeec9ccb0e1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:46.796780 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ae6b7-cd3f-4618-950e-baeec9ccb0e1 service placement] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:46.808681 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1441/2875] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:04:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:04:49.925722 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d62313-33ef-44a2-bc31-024d3086b7b3 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:49.936229 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-d2d62313-33ef-44a2-bc31-024d3086b7b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:49.940891 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d62313-33ef-44a2-bc31-024d3086b7b3 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:49.945719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d20155-b50f-4e7e-8b61-6a4d036f20b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:49.946487 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d20155-b50f-4e7e-8b61-6a4d036f20b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:49.946839 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d20155-b50f-4e7e-8b61-6a4d036f20b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:49.947234 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d20155-b50f-4e7e-8b61-6a4d036f20b6 admin admin] PATH_INFO: `/v3/users/d69d5ca56a8e43ca9d91920684f2dac8` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:49.948820 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d20155-b50f-4e7e-8b61-6a4d036f20b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d69d5ca56a8e43ca9d91920684f2dac8)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:49.950947 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d20155-b50f-4e7e-8b61-6a4d036f20b6 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:50.002933 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-73d20155-b50f-4e7e-8b61-6a4d036f20b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd69d5ca56a8e43ca9d91920684f2dac8'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:04:50.014924 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-73d20155-b50f-4e7e-8b61-6a4d036f20b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd69d5ca56a8e43ca9d91920684f2dac8'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:04:50.022718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-73d20155-b50f-4e7e-8b61-6a4d036f20b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd69d5ca56a8e43ca9d91920684f2dac8'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:04:50.030185 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-73d20155-b50f-4e7e-8b61-6a4d036f20b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd69d5ca56a8e43ca9d91920684f2dac8'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:04:50.031417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-73d20155-b50f-4e7e-8b61-6a4d036f20b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd69d5ca56a8e43ca9d91920684f2dac8'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:04:50.037342 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1435/2876] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 01:04:49 2022] DELETE /identity/v3/users/d69d5ca56a8e43ca9d91920684f2dac8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:04:50.214893 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ae6b7-cd3f-4618-950e-baeec9ccb0e1 service placement] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:50.227274 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ae6b7-cd3f-4618-950e-baeec9ccb0e1 service placement] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:50.285525 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a46da-dadb-4bd2-80db-5ba9c025a768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:50.286150 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a46da-dadb-4bd2-80db-5ba9c025a768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:50.286426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a46da-dadb-4bd2-80db-5ba9c025a768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:50.286697 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a46da-dadb-4bd2-80db-5ba9c025a768 admin admin] PATH_INFO: `/v3/projects/e5fdc920045d4fabac1e51adb944ff3f` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:50.287838 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a46da-dadb-4bd2-80db-5ba9c025a768 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5fdc920045d4fabac1e51adb944ff3f)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:50.289747 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a46da-dadb-4bd2-80db-5ba9c025a768 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:50.322550 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:50.336743 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-9a7a46da-dadb-4bd2-80db-5ba9c025a768 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5fdc920045d4fabac1e51adb944ff3f', 'request_id': 'req-9a7a46da-dadb-4bd2-80db-5ba9c025a768'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:04:50.342210 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-9a7a46da-dadb-4bd2-80db-5ba9c025a768 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5fdc920045d4fabac1e51adb944ff3f', 'request_id': 'req-9a7a46da-dadb-4bd2-80db-5ba9c025a768'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:04:50.348902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-9a7a46da-dadb-4bd2-80db-5ba9c025a768 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5fdc920045d4fabac1e51adb944ff3f', 'request_id': 'req-9a7a46da-dadb-4bd2-80db-5ba9c025a768'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:04:50.361634 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-9a7a46da-dadb-4bd2-80db-5ba9c025a768 admin admin] The token cache is being invalidate because project e5fdc920045d4fabac1e51adb944ff3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:04:50.361923 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-9a7a46da-dadb-4bd2-80db-5ba9c025a768 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:04:50.362540 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-9a7a46da-dadb-4bd2-80db-5ba9c025a768 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:04:50.364044 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1442/2877] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 01:04:50 2022] DELETE /identity/v3/projects/e5fdc920045d4fabac1e51adb944ff3f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:04:50.368096 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d20155-b50f-4e7e-8b61-6a4d036f20b6 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:50.369495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-73d20155-b50f-4e7e-8b61-6a4d036f20b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:50.383145 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d20155-b50f-4e7e-8b61-6a4d036f20b6 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:50.440086 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b339edf-5590-4347-a049-3a063c28281f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:50.440642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b339edf-5590-4347-a049-3a063c28281f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:50.440922 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b339edf-5590-4347-a049-3a063c28281f admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:50.441197 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b339edf-5590-4347-a049-3a063c28281f admin admin] PATH_INFO: `/v3/users/5f99a7b91cd54ba7a9bd79484a7fb436` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:50.442326 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b339edf-5590-4347-a049-3a063c28281f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f99a7b91cd54ba7a9bd79484a7fb436)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:50.444294 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b339edf-5590-4347-a049-3a063c28281f admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:50.449586 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:04:50.491688 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-1b339edf-5590-4347-a049-3a063c28281f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f99a7b91cd54ba7a9bd79484a7fb436'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:04:50.504340 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-1b339edf-5590-4347-a049-3a063c28281f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f99a7b91cd54ba7a9bd79484a7fb436'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:04:50.511065 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-1b339edf-5590-4347-a049-3a063c28281f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f99a7b91cd54ba7a9bd79484a7fb436'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:04:50.519550 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-1b339edf-5590-4347-a049-3a063c28281f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f99a7b91cd54ba7a9bd79484a7fb436'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:04:50.520153 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-1b339edf-5590-4347-a049-3a063c28281f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f99a7b91cd54ba7a9bd79484a7fb436'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:04:50.525949 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1436/2878] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 01:04:50 2022] DELETE /identity/v3/users/5f99a7b91cd54ba7a9bd79484a7fb436 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:04:50.700284 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a46da-dadb-4bd2-80db-5ba9c025a768 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:04:50.702090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-9a7a46da-dadb-4bd2-80db-5ba9c025a768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:04:50.715016 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a46da-dadb-4bd2-80db-5ba9c025a768 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:04:50.771914 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92934d80-f8bc-496c-b131-26190e696e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:04:50.772542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92934d80-f8bc-496c-b131-26190e696e62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:04:50.772820 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92934d80-f8bc-496c-b131-26190e696e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:04:50.773089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92934d80-f8bc-496c-b131-26190e696e62 admin admin] PATH_INFO: `/v3/projects/4a00d4bba4a2464f8880c7b4de32bd54` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:04:50.774193 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92934d80-f8bc-496c-b131-26190e696e62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a00d4bba4a2464f8880c7b4de32bd54)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:04:50.776121 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92934d80-f8bc-496c-b131-26190e696e62 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:04:50.822550 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-92934d80-f8bc-496c-b131-26190e696e62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a00d4bba4a2464f8880c7b4de32bd54', 'request_id': 'req-92934d80-f8bc-496c-b131-26190e696e62'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:04:50.827974 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-92934d80-f8bc-496c-b131-26190e696e62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a00d4bba4a2464f8880c7b4de32bd54', 'request_id': 'req-92934d80-f8bc-496c-b131-26190e696e62'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:04:50.834047 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-92934d80-f8bc-496c-b131-26190e696e62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a00d4bba4a2464f8880c7b4de32bd54', 'request_id': 'req-92934d80-f8bc-496c-b131-26190e696e62'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:04:50.850510 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-92934d80-f8bc-496c-b131-26190e696e62 admin admin] The token cache is being invalidate because project 4a00d4bba4a2464f8880c7b4de32bd54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:04:50.850831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-92934d80-f8bc-496c-b131-26190e696e62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:04:50.852004 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-92934d80-f8bc-496c-b131-26190e696e62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:04:50.853484 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1443/2879] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 01:04:50 2022] DELETE /identity/v3/projects/4a00d4bba4a2464f8880c7b4de32bd54 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:05:00.522554 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:00.847448 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=64,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:15.920423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b339edf-5590-4347-a049-3a063c28281f admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:15.922277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-1b339edf-5590-4347-a049-3a063c28281f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:16.011647 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b339edf-5590-4347-a049-3a063c28281f admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:16.072769 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c66c76-ad99-4684-9e5a-8e68027b9c61 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdacdcb2451742938164fdd44cdbaebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:16.073437 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c66c76-ad99-4684-9e5a-8e68027b9c61 service placement] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:16.073747 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c66c76-ad99-4684-9e5a-8e68027b9c61 service placement] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:16.074022 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c66c76-ad99-4684-9e5a-8e68027b9c61 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:16.075773 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-57c66c76-ad99-4684-9e5a-8e68027b9c61 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:16.088127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c66c76-ad99-4684-9e5a-8e68027b9c61 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:16.089697 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c66c76-ad99-4684-9e5a-8e68027b9c61 service placement] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:16.103143 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1437/2880] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:05:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:05:19.663427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666645a-331a-4608-aaaf-e0ab81aa9027 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:19.663906 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666645a-331a-4608-aaaf-e0ab81aa9027 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:19.664010 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666645a-331a-4608-aaaf-e0ab81aa9027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:19.683793 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-d666645a-331a-4608-aaaf-e0ab81aa9027 None None] MFA Rules not processed for user `26826ddf92124c95835a5356e13040bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:05:19.684975 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-d666645a-331a-4608-aaaf-e0ab81aa9027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:19.739820 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1444/2881] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:05:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:05:20.365539 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c66c76-ad99-4684-9e5a-8e68027b9c61 service placement] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:05:20.367060 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-57c66c76-ad99-4684-9e5a-8e68027b9c61 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:20.380995 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c66c76-ad99-4684-9e5a-8e68027b9c61 service placement] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:05:20.386133 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d714a5d-c96c-408b-b7bd-7c3be64e4878 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdacdcb2451742938164fdd44cdbaebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:05:20.386674 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d714a5d-c96c-408b-b7bd-7c3be64e4878 service placement] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:05:20.386953 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d714a5d-c96c-408b-b7bd-7c3be64e4878 service placement] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:05:20.387248 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d714a5d-c96c-408b-b7bd-7c3be64e4878 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:05:20.388956 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-7d714a5d-c96c-408b-b7bd-7c3be64e4878 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:05:20.401535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d714a5d-c96c-408b-b7bd-7c3be64e4878 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:05:20.403169 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d714a5d-c96c-408b-b7bd-7c3be64e4878 service placement] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:05:20.418015 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1438/2882] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:05:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:05:29.732202 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=25 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:05:30.409269 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=68 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:06:50.631737 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab37433-336a-4d88-884e-5734d16f1496 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:50.632247 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab37433-336a-4d88-884e-5734d16f1496 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:50.632339 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab37433-336a-4d88-884e-5734d16f1496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:50.655861 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-4ab37433-336a-4d88-884e-5734d16f1496 None None] MFA Rules not processed for user `a8fb7a9acefd4b418c992661037c2ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:06:50.657059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4ab37433-336a-4d88-884e-5734d16f1496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:06:50.716870 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1445/2883] 10.222.0.67 () {58 vars in 967 bytes} [Wed Aug 31 01:06:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:06:50.757543 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d714a5d-c96c-408b-b7bd-7c3be64e4878 service placement] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:50.758979 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-7d714a5d-c96c-408b-b7bd-7c3be64e4878 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:06:50.830852 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d714a5d-c96c-408b-b7bd-7c3be64e4878 service placement] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:50.835851 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba07df-bef0-4cfc-9966-7da4ec03d523 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd3cfbc8ebd4c5a83bef09efe10bc91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:50.836453 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba07df-bef0-4cfc-9966-7da4ec03d523 service swift] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:50.836733 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba07df-bef0-4cfc-9966-7da4ec03d523 service swift] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:50.837023 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba07df-bef0-4cfc-9966-7da4ec03d523 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:50.846196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba07df-bef0-4cfc-9966-7da4ec03d523 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:06:50.847804 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba07df-bef0-4cfc-9966-7da4ec03d523 service swift] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:06:50.855977 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1439/2884] 10.222.0.67 () {60 vars in 1384 bytes} [Wed Aug 31 01:06:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 01:06:54.323678 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab37433-336a-4d88-884e-5734d16f1496 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:54.326017 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4ab37433-336a-4d88-884e-5734d16f1496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:06:54.341822 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab37433-336a-4d88-884e-5734d16f1496 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:54.347217 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9b60d-f9a0-496a-bc36-bc709a3971a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdacdcb2451742938164fdd44cdbaebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:54.347830 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9b60d-f9a0-496a-bc36-bc709a3971a3 service placement] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:54.348134 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9b60d-f9a0-496a-bc36-bc709a3971a3 service placement] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:54.348368 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9b60d-f9a0-496a-bc36-bc709a3971a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:54.350252 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-7ea9b60d-f9a0-496a-bc36-bc709a3971a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:06:54.368382 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9b60d-f9a0-496a-bc36-bc709a3971a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:06:54.368382 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9b60d-f9a0-496a-bc36-bc709a3971a3 service placement] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:06:54.383773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1446/2885] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:06:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:06:57.621824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba07df-bef0-4cfc-9966-7da4ec03d523 service swift] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:57.623434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-46ba07df-bef0-4cfc-9966-7da4ec03d523 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:06:57.694069 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba07df-bef0-4cfc-9966-7da4ec03d523 service swift] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:57.698594 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078be2bc-57a1-4059-aab8-5776ced5fc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:57.699281 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078be2bc-57a1-4059-aab8-5776ced5fc31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:57.699798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078be2bc-57a1-4059-aab8-5776ced5fc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:57.700138 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078be2bc-57a1-4059-aab8-5776ced5fc31 admin admin] PATH_INFO: `/v3/users/74ec301c62e541e0851f7570bcf07e4f` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:57.701275 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078be2bc-57a1-4059-aab8-5776ced5fc31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74ec301c62e541e0851f7570bcf07e4f)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:06:57.703217 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078be2bc-57a1-4059-aab8-5776ced5fc31 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:06:57.750037 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-078be2bc-57a1-4059-aab8-5776ced5fc31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74ec301c62e541e0851f7570bcf07e4f'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:06:57.762341 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-078be2bc-57a1-4059-aab8-5776ced5fc31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74ec301c62e541e0851f7570bcf07e4f'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:06:57.770212 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-078be2bc-57a1-4059-aab8-5776ced5fc31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74ec301c62e541e0851f7570bcf07e4f'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:06:57.780474 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-078be2bc-57a1-4059-aab8-5776ced5fc31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74ec301c62e541e0851f7570bcf07e4f'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:06:57.781543 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-078be2bc-57a1-4059-aab8-5776ced5fc31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74ec301c62e541e0851f7570bcf07e4f'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:06:57.788387 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1440/2886] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 01:06:57 2022] DELETE /identity/v3/users/74ec301c62e541e0851f7570bcf07e4f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:06:57.970445 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9b60d-f9a0-496a-bc36-bc709a3971a3 service placement] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:57.981904 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9b60d-f9a0-496a-bc36-bc709a3971a3 service placement] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:58.041090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc602c58-b6b5-4dfa-a9ad-107094b90078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:58.041718 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc602c58-b6b5-4dfa-a9ad-107094b90078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:58.042052 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc602c58-b6b5-4dfa-a9ad-107094b90078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:58.042273 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc602c58-b6b5-4dfa-a9ad-107094b90078 admin admin] PATH_INFO: `/v3/projects/7f85c7d484fa4aebafc3c0a12c9569c2` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:58.043625 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc602c58-b6b5-4dfa-a9ad-107094b90078 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f85c7d484fa4aebafc3c0a12c9569c2)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:06:58.045513 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc602c58-b6b5-4dfa-a9ad-107094b90078 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:06:58.092235 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-cc602c58-b6b5-4dfa-a9ad-107094b90078 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f85c7d484fa4aebafc3c0a12c9569c2', 'request_id': 'req-cc602c58-b6b5-4dfa-a9ad-107094b90078'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:06:58.098341 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-cc602c58-b6b5-4dfa-a9ad-107094b90078 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f85c7d484fa4aebafc3c0a12c9569c2', 'request_id': 'req-cc602c58-b6b5-4dfa-a9ad-107094b90078'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:06:58.104283 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-cc602c58-b6b5-4dfa-a9ad-107094b90078 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f85c7d484fa4aebafc3c0a12c9569c2', 'request_id': 'req-cc602c58-b6b5-4dfa-a9ad-107094b90078'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:06:58.116890 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-cc602c58-b6b5-4dfa-a9ad-107094b90078 admin admin] The token cache is being invalidate because project 7f85c7d484fa4aebafc3c0a12c9569c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:06:58.117180 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-cc602c58-b6b5-4dfa-a9ad-107094b90078 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:06:58.117819 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-cc602c58-b6b5-4dfa-a9ad-107094b90078 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:06:58.119302 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1447/2887] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 01:06:57 2022] DELETE /identity/v3/projects/7f85c7d484fa4aebafc3c0a12c9569c2 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:06:58.124041 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078be2bc-57a1-4059-aab8-5776ced5fc31 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:58.125394 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-078be2bc-57a1-4059-aab8-5776ced5fc31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:06:58.138905 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078be2bc-57a1-4059-aab8-5776ced5fc31 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:58.143442 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:06:58.197253 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12aed7-f468-4dbd-95de-74cdd28baee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:58.197876 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12aed7-f468-4dbd-95de-74cdd28baee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:58.198150 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12aed7-f468-4dbd-95de-74cdd28baee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:58.198419 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12aed7-f468-4dbd-95de-74cdd28baee0 admin admin] PATH_INFO: `/v3/users/4e21a053d6664863a65972e90326935d` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:58.199631 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12aed7-f468-4dbd-95de-74cdd28baee0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e21a053d6664863a65972e90326935d)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:06:58.201529 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12aed7-f468-4dbd-95de-74cdd28baee0 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:06:58.250644 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ea12aed7-f468-4dbd-95de-74cdd28baee0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e21a053d6664863a65972e90326935d'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:06:58.263145 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ea12aed7-f468-4dbd-95de-74cdd28baee0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e21a053d6664863a65972e90326935d'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:06:58.270003 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ea12aed7-f468-4dbd-95de-74cdd28baee0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e21a053d6664863a65972e90326935d'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:06:58.277432 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ea12aed7-f468-4dbd-95de-74cdd28baee0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e21a053d6664863a65972e90326935d'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:06:58.278299 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-ea12aed7-f468-4dbd-95de-74cdd28baee0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e21a053d6664863a65972e90326935d'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:06:58.284174 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1441/2888] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 01:06:58 2022] DELETE /identity/v3/users/4e21a053d6664863a65972e90326935d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:06:58.467223 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc602c58-b6b5-4dfa-a9ad-107094b90078 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:06:58.468863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-cc602c58-b6b5-4dfa-a9ad-107094b90078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:06:58.486583 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc602c58-b6b5-4dfa-a9ad-107094b90078 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:06:58.506125 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:06:58.546699 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d89d8-719c-4c56-b88a-774a08d3d3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:06:58.547431 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d89d8-719c-4c56-b88a-774a08d3d3b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:06:58.547708 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d89d8-719c-4c56-b88a-774a08d3d3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:06:58.547976 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d89d8-719c-4c56-b88a-774a08d3d3b0 admin admin] PATH_INFO: `/v3/projects/b21db527ca88436082ea46e41ac5f545` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:06:58.549110 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d89d8-719c-4c56-b88a-774a08d3d3b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b21db527ca88436082ea46e41ac5f545)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:06:58.551060 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d89d8-719c-4c56-b88a-774a08d3d3b0 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:06:58.598205 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-fd7d89d8-719c-4c56-b88a-774a08d3d3b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b21db527ca88436082ea46e41ac5f545', 'request_id': 'req-fd7d89d8-719c-4c56-b88a-774a08d3d3b0'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:06:58.603444 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-fd7d89d8-719c-4c56-b88a-774a08d3d3b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b21db527ca88436082ea46e41ac5f545', 'request_id': 'req-fd7d89d8-719c-4c56-b88a-774a08d3d3b0'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:06:58.609975 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-fd7d89d8-719c-4c56-b88a-774a08d3d3b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b21db527ca88436082ea46e41ac5f545', 'request_id': 'req-fd7d89d8-719c-4c56-b88a-774a08d3d3b0'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:06:58.623155 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-fd7d89d8-719c-4c56-b88a-774a08d3d3b0 admin admin] The token cache is being invalidate because project b21db527ca88436082ea46e41ac5f545 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81472) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:06:58.623453 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-fd7d89d8-719c-4c56-b88a-774a08d3d3b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:06:58.624109 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-fd7d89d8-719c-4c56-b88a-774a08d3d3b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:06:58.625593 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1448/2889] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 01:06:58 2022] DELETE /identity/v3/projects/b21db527ca88436082ea46e41ac5f545 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:08.280840 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:08.619939 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:22.223227 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12aed7-f468-4dbd-95de-74cdd28baee0 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:22.224645 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-ea12aed7-f468-4dbd-95de-74cdd28baee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:22.294484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12aed7-f468-4dbd-95de-74cdd28baee0 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:22.342786 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016173f-47e0-4b42-b81d-b628b3485fc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:22.343414 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016173f-47e0-4b42-b81d-b628b3485fc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:22.343692 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016173f-47e0-4b42-b81d-b628b3485fc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:22.343962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016173f-47e0-4b42-b81d-b628b3485fc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:22.345718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-c016173f-47e0-4b42-b81d-b628b3485fc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:22.408862 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016173f-47e0-4b42-b81d-b628b3485fc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:22.410207 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016173f-47e0-4b42-b81d-b628b3485fc8 service neutron] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:22.419647 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1442/2890] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:07:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:07:25.608415 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d89d8-719c-4c56-b88a-774a08d3d3b0 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:25.610328 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-fd7d89d8-719c-4c56-b88a-774a08d3d3b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:25.626197 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d89d8-719c-4c56-b88a-774a08d3d3b0 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:25.630897 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b62b4f-5ce1-4e8d-bf1f-168d0fbdd54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:25.631573 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b62b4f-5ce1-4e8d-bf1f-168d0fbdd54a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:25.631859 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b62b4f-5ce1-4e8d-bf1f-168d0fbdd54a admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:25.632136 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b62b4f-5ce1-4e8d-bf1f-168d0fbdd54a admin admin] PATH_INFO: `/v3/users/26826ddf92124c95835a5356e13040bf` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:25.633285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b62b4f-5ce1-4e8d-bf1f-168d0fbdd54a admin admin] RBAC: Authorizing `identity:delete_user(user_id=26826ddf92124c95835a5356e13040bf)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:25.635241 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b62b4f-5ce1-4e8d-bf1f-168d0fbdd54a admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:25.689569 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-e1b62b4f-5ce1-4e8d-bf1f-168d0fbdd54a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26826ddf92124c95835a5356e13040bf'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:25.697836 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-e1b62b4f-5ce1-4e8d-bf1f-168d0fbdd54a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26826ddf92124c95835a5356e13040bf'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:25.707531 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-e1b62b4f-5ce1-4e8d-bf1f-168d0fbdd54a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26826ddf92124c95835a5356e13040bf'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:25.708416 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-e1b62b4f-5ce1-4e8d-bf1f-168d0fbdd54a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26826ddf92124c95835a5356e13040bf'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:25.709031 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-e1b62b4f-5ce1-4e8d-bf1f-168d0fbdd54a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26826ddf92124c95835a5356e13040bf'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:25.730324 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1449/2891] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 01:07:25 2022] DELETE /identity/v3/users/26826ddf92124c95835a5356e13040bf => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:25.920352 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016173f-47e0-4b42-b81d-b628b3485fc8 service neutron] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:25.930487 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016173f-47e0-4b42-b81d-b628b3485fc8 service neutron] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:25.984697 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=100 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:25.998061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9dad6-b060-42b1-b05b-b67b2c3b480b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:25.999606 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9dad6-b060-42b1-b05b-b67b2c3b480b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:26.000167 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9dad6-b060-42b1-b05b-b67b2c3b480b admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:26.000677 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9dad6-b060-42b1-b05b-b67b2c3b480b admin admin] PATH_INFO: `/v3/projects/6b67e536fd414591b859c1a89808e1a6` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:26.001908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9dad6-b060-42b1-b05b-b67b2c3b480b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b67e536fd414591b859c1a89808e1a6)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:26.003960 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9dad6-b060-42b1-b05b-b67b2c3b480b admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:26.055355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-57c9dad6-b060-42b1-b05b-b67b2c3b480b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b67e536fd414591b859c1a89808e1a6', 'request_id': 'req-57c9dad6-b060-42b1-b05b-b67b2c3b480b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:26.063789 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-57c9dad6-b060-42b1-b05b-b67b2c3b480b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b67e536fd414591b859c1a89808e1a6', 'request_id': 'req-57c9dad6-b060-42b1-b05b-b67b2c3b480b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:26.069685 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-57c9dad6-b060-42b1-b05b-b67b2c3b480b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b67e536fd414591b859c1a89808e1a6', 'request_id': 'req-57c9dad6-b060-42b1-b05b-b67b2c3b480b'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:26.081555 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-57c9dad6-b060-42b1-b05b-b67b2c3b480b admin admin] The token cache is being invalidate because project 6b67e536fd414591b859c1a89808e1a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:07:26.081850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-57c9dad6-b060-42b1-b05b-b67b2c3b480b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:26.082499 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-57c9dad6-b060-42b1-b05b-b67b2c3b480b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:26.084006 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1443/2892] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 01:07:25 2022] DELETE /identity/v3/projects/6b67e536fd414591b859c1a89808e1a6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:07:26.089059 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f19f4-7b0e-451b-93b9-f63150f69a16 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:26.089345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f19f4-7b0e-451b-93b9-f63150f69a16 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:26.089614 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f19f4-7b0e-451b-93b9-f63150f69a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:26.109277 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-d17f19f4-7b0e-451b-93b9-f63150f69a16 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:26.110459 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-d17f19f4-7b0e-451b-93b9-f63150f69a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:26.169639 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1450/2893] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:07:26.178404 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706747f-355b-41c1-abb7-68f897744f89 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:26.178692 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706747f-355b-41c1-abb7-68f897744f89 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:26.178971 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706747f-355b-41c1-abb7-68f897744f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:26.199516 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-9706747f-355b-41c1-abb7-68f897744f89 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:26.200707 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-9706747f-355b-41c1-abb7-68f897744f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:26.209445 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1444/2894] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:07:26.213705 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f19f4-7b0e-451b-93b9-f63150f69a16 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:26.224080 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f19f4-7b0e-451b-93b9-f63150f69a16 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:26.228744 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab16f1-788a-4ae6-8636-930afda2b4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:26.229339 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab16f1-788a-4ae6-8636-930afda2b4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:26.229622 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab16f1-788a-4ae6-8636-930afda2b4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:26.229897 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab16f1-788a-4ae6-8636-930afda2b4d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:26.230537 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab16f1-788a-4ae6-8636-930afda2b4d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:26.230842 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab16f1-788a-4ae6-8636-930afda2b4d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:26.232478 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab16f1-788a-4ae6-8636-930afda2b4d3 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:26.244076 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1451/2895] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 01:07:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:26.249154 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706747f-355b-41c1-abb7-68f897744f89 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:26.258902 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706747f-355b-41c1-abb7-68f897744f89 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:26.263545 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2733d8e-e28c-4af7-8dd1-4a195d438ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:26.264672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2733d8e-e28c-4af7-8dd1-4a195d438ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:26.265114 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2733d8e-e28c-4af7-8dd1-4a195d438ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:26.265269 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2733d8e-e28c-4af7-8dd1-4a195d438ae4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:26.265878 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2733d8e-e28c-4af7-8dd1-4a195d438ae4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:26.267775 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2733d8e-e28c-4af7-8dd1-4a195d438ae4 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:26.297347 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1445/2896] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 01:07:26 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:07:26.302054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab16f1-788a-4ae6-8636-930afda2b4d3 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:26.312855 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab16f1-788a-4ae6-8636-930afda2b4d3 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:26.371560 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95872d22-9c98-4cc1-97a2-a8b31c48f860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:26.372542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95872d22-9c98-4cc1-97a2-a8b31c48f860 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:26.373025 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95872d22-9c98-4cc1-97a2-a8b31c48f860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:26.374090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95872d22-9c98-4cc1-97a2-a8b31c48f860 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:26.375258 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95872d22-9c98-4cc1-97a2-a8b31c48f860 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:26.378430 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95872d22-9c98-4cc1-97a2-a8b31c48f860 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:26.387390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:26.402390 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1452/2897] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 01:07:26 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:07:26.407776 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2733d8e-e28c-4af7-8dd1-4a195d438ae4 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:26.419422 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2733d8e-e28c-4af7-8dd1-4a195d438ae4 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:26.424810 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b6483-ecfa-431f-8007-8ec34ebdf34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:26.425349 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b6483-ecfa-431f-8007-8ec34ebdf34e admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:26.425621 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b6483-ecfa-431f-8007-8ec34ebdf34e admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:26.425885 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b6483-ecfa-431f-8007-8ec34ebdf34e admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:26.426513 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b6483-ecfa-431f-8007-8ec34ebdf34e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:26.428153 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b6483-ecfa-431f-8007-8ec34ebdf34e admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:26.438892 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1446/2898] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:07:26.442845 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95872d22-9c98-4cc1-97a2-a8b31c48f860 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:26.453461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95872d22-9c98-4cc1-97a2-a8b31c48f860 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:26.458071 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ff50e-92ca-45fe-98fc-e088022bc40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:26.458663 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ff50e-92ca-45fe-98fc-e088022bc40b admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:26.458938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ff50e-92ca-45fe-98fc-e088022bc40b admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:26.459240 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ff50e-92ca-45fe-98fc-e088022bc40b admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:26.459873 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ff50e-92ca-45fe-98fc-e088022bc40b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:26.461447 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ff50e-92ca-45fe-98fc-e088022bc40b admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:26.472902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1453/2899] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:07:26.476997 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b6483-ecfa-431f-8007-8ec34ebdf34e admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:26.487433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b6483-ecfa-431f-8007-8ec34ebdf34e admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:26.492187 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ffb2b-6d52-4400-add2-85a5370b5f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:26.492829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ffb2b-6d52-4400-add2-85a5370b5f44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:26.493107 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ffb2b-6d52-4400-add2-85a5370b5f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:26.493380 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ffb2b-6d52-4400-add2-85a5370b5f44 admin admin] PATH_INFO: `/v3/projects/be57e81a767549cda9c3f913eb8c02fc/users/678dcce1cb964238888c27efb68689c3/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:26.507840 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ffb2b-6d52-4400-add2-85a5370b5f44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be57e81a767549cda9c3f913eb8c02fc, user_id=678dcce1cb964238888c27efb68689c3, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:26.510514 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ffb2b-6d52-4400-add2-85a5370b5f44 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:26.524087 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1447/2900] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 01:07:26 2022] PUT /identity/v3/projects/be57e81a767549cda9c3f913eb8c02fc/users/678dcce1cb964238888c27efb68689c3/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:26.534352 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254df63-3b86-4be8-a14a-728f852f2c7a None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:26.534640 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254df63-3b86-4be8-a14a-728f852f2c7a None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:26.534915 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254df63-3b86-4be8-a14a-728f852f2c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:26.561726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-2254df63-3b86-4be8-a14a-728f852f2c7a None None] MFA Rules not processed for user `678dcce1cb964238888c27efb68689c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:26.562930 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-2254df63-3b86-4be8-a14a-728f852f2c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:26.628739 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1454/2901] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:07:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:07:26.641738 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ffb2b-6d52-4400-add2-85a5370b5f44 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:26.643536 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-e29ffb2b-6d52-4400-add2-85a5370b5f44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:26.727083 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ffb2b-6d52-4400-add2-85a5370b5f44 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:26.732569 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deabb09-d7dd-4ade-8676-e3a4579e1357 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aa53ae5a30146f69d7d6eefe4290dba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:26.733277 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deabb09-d7dd-4ade-8676-e3a4579e1357 service nova] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:26.733554 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deabb09-d7dd-4ade-8676-e3a4579e1357 service nova] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:26.733821 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deabb09-d7dd-4ade-8676-e3a4579e1357 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:26.741323 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deabb09-d7dd-4ade-8676-e3a4579e1357 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:26.742886 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deabb09-d7dd-4ade-8676-e3a4579e1357 service nova] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:26.752332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1448/2902] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:07:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:07:36.145214 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2254df63-3b86-4be8-a14a-728f852f2c7a None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:36.206866 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2254df63-3b86-4be8-a14a-728f852f2c7a None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:36.211350 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e46a4b-01be-4d9f-a6ea-ffa73a7ec459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:36.211977 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e46a4b-01be-4d9f-a6ea-ffa73a7ec459 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:36.212255 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e46a4b-01be-4d9f-a6ea-ffa73a7ec459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:36.212524 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e46a4b-01be-4d9f-a6ea-ffa73a7ec459 admin admin] PATH_INFO: `/v3/users/678dcce1cb964238888c27efb68689c3` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:36.213632 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e46a4b-01be-4d9f-a6ea-ffa73a7ec459 admin admin] RBAC: Authorizing `identity:delete_user(user_id=678dcce1cb964238888c27efb68689c3)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:36.215585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e46a4b-01be-4d9f-a6ea-ffa73a7ec459 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:36.262243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-98e46a4b-01be-4d9f-a6ea-ffa73a7ec459 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '678dcce1cb964238888c27efb68689c3'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:36.270828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-98e46a4b-01be-4d9f-a6ea-ffa73a7ec459 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '678dcce1cb964238888c27efb68689c3'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:36.277649 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-98e46a4b-01be-4d9f-a6ea-ffa73a7ec459 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '678dcce1cb964238888c27efb68689c3'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:36.278293 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-98e46a4b-01be-4d9f-a6ea-ffa73a7ec459 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '678dcce1cb964238888c27efb68689c3'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:36.278891 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-98e46a4b-01be-4d9f-a6ea-ffa73a7ec459 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '678dcce1cb964238888c27efb68689c3'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:36.285023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:36.299584 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1455/2903] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 01:07:36 2022] DELETE /identity/v3/users/678dcce1cb964238888c27efb68689c3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:36.304035 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f1599-f9f0-4cc9-a019-a30b99f2ec93 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:36.304542 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f1599-f9f0-4cc9-a019-a30b99f2ec93 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:36.304637 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f1599-f9f0-4cc9-a019-a30b99f2ec93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:36.318472 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:36.324870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-686f1599-f9f0-4cc9-a019-a30b99f2ec93 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:36.327183 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-686f1599-f9f0-4cc9-a019-a30b99f2ec93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:36.389652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1449/2904] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:07:36.399642 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e46a4b-01be-4d9f-a6ea-ffa73a7ec459 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:36.401053 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-98e46a4b-01be-4d9f-a6ea-ffa73a7ec459 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:36.460487 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e46a4b-01be-4d9f-a6ea-ffa73a7ec459 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:36.464023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf2735-48aa-4b9e-b5d3-c92b8b86b0f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:36.464621 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf2735-48aa-4b9e-b5d3-c92b8b86b0f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:36.464904 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf2735-48aa-4b9e-b5d3-c92b8b86b0f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:36.465175 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf2735-48aa-4b9e-b5d3-c92b8b86b0f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:36.475016 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf2735-48aa-4b9e-b5d3-c92b8b86b0f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:36.476474 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf2735-48aa-4b9e-b5d3-c92b8b86b0f6 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:36.486556 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1456/2905] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:07:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 01:07:36.759908 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f1599-f9f0-4cc9-a019-a30b99f2ec93 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:36.761359 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-686f1599-f9f0-4cc9-a019-a30b99f2ec93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:36.774593 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f1599-f9f0-4cc9-a019-a30b99f2ec93 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:36.779260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d76fc9-bd1d-432a-a078-f1612308cd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:36.780134 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d76fc9-bd1d-432a-a078-f1612308cd74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:36.780416 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d76fc9-bd1d-432a-a078-f1612308cd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:36.780753 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d76fc9-bd1d-432a-a078-f1612308cd74 admin admin] PATH_INFO: `/v3/projects/be57e81a767549cda9c3f913eb8c02fc` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:36.781893 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d76fc9-bd1d-432a-a078-f1612308cd74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be57e81a767549cda9c3f913eb8c02fc)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:36.783919 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d76fc9-bd1d-432a-a078-f1612308cd74 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:36.832081 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-53d76fc9-bd1d-432a-a078-f1612308cd74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be57e81a767549cda9c3f913eb8c02fc', 'request_id': 'req-53d76fc9-bd1d-432a-a078-f1612308cd74'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:36.839336 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-53d76fc9-bd1d-432a-a078-f1612308cd74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be57e81a767549cda9c3f913eb8c02fc', 'request_id': 'req-53d76fc9-bd1d-432a-a078-f1612308cd74'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:36.845354 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-53d76fc9-bd1d-432a-a078-f1612308cd74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be57e81a767549cda9c3f913eb8c02fc', 'request_id': 'req-53d76fc9-bd1d-432a-a078-f1612308cd74'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:36.856467 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-53d76fc9-bd1d-432a-a078-f1612308cd74 admin admin] The token cache is being invalidate because project be57e81a767549cda9c3f913eb8c02fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:07:36.856754 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-53d76fc9-bd1d-432a-a078-f1612308cd74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:36.857389 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-53d76fc9-bd1d-432a-a078-f1612308cd74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:36.859641 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1450/2906] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 01:07:36 2022] DELETE /identity/v3/projects/be57e81a767549cda9c3f913eb8c02fc => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:36.864684 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84089c-18ba-4924-ae52-0ceef2eaab56 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:36.864969 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84089c-18ba-4924-ae52-0ceef2eaab56 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:36.865241 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84089c-18ba-4924-ae52-0ceef2eaab56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:36.885148 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-4f84089c-18ba-4924-ae52-0ceef2eaab56 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:36.886381 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4f84089c-18ba-4924-ae52-0ceef2eaab56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:36.945434 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1457/2907] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:07:36.954809 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da20de-5905-4fca-bf9f-3ce6c0f43f95 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:36.955155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da20de-5905-4fca-bf9f-3ce6c0f43f95 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:36.955441 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da20de-5905-4fca-bf9f-3ce6c0f43f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:36.976106 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-19da20de-5905-4fca-bf9f-3ce6c0f43f95 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:36.977303 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-19da20de-5905-4fca-bf9f-3ce6c0f43f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:36.985471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1451/2908] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:07:36.989860 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f84089c-18ba-4924-ae52-0ceef2eaab56 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:36.999989 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f84089c-18ba-4924-ae52-0ceef2eaab56 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:37.004912 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf6e58-c72d-48af-a5ef-ccde7b4c597f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:37.005440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf6e58-c72d-48af-a5ef-ccde7b4c597f admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:37.005732 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf6e58-c72d-48af-a5ef-ccde7b4c597f admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:37.006261 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf6e58-c72d-48af-a5ef-ccde7b4c597f admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:37.007106 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcf6e58-c72d-48af-a5ef-ccde7b4c597f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:37.007420 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcf6e58-c72d-48af-a5ef-ccde7b4c597f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:37.009033 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcf6e58-c72d-48af-a5ef-ccde7b4c597f admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:37.021549 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1458/2909] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 01:07:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:07:37.026100 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da20de-5905-4fca-bf9f-3ce6c0f43f95 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:37.035946 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da20de-5905-4fca-bf9f-3ce6c0f43f95 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:37.040646 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70ab87-3d2c-442d-8d00-7ce4a4ad8baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:37.041240 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70ab87-3d2c-442d-8d00-7ce4a4ad8baf admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:37.041519 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70ab87-3d2c-442d-8d00-7ce4a4ad8baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:37.041788 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70ab87-3d2c-442d-8d00-7ce4a4ad8baf admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:37.042417 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa70ab87-3d2c-442d-8d00-7ce4a4ad8baf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:37.044406 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa70ab87-3d2c-442d-8d00-7ce4a4ad8baf admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:37.071401 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1452/2910] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 01:07:37 2022] POST /identity/v3/projects => generated 361 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:07:37.075546 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf6e58-c72d-48af-a5ef-ccde7b4c597f admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:37.086157 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf6e58-c72d-48af-a5ef-ccde7b4c597f admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:37.144920 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b60ad-4482-44fc-ad55-4d577df6112d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:37.145526 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b60ad-4482-44fc-ad55-4d577df6112d admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:37.146080 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b60ad-4482-44fc-ad55-4d577df6112d admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:37.146080 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b60ad-4482-44fc-ad55-4d577df6112d admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:37.146676 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b60ad-4482-44fc-ad55-4d577df6112d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:37.148646 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b60ad-4482-44fc-ad55-4d577df6112d admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:37.169933 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1459/2911] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 01:07:37 2022] POST /identity/v3/users => generated 281 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:07:37.174353 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70ab87-3d2c-442d-8d00-7ce4a4ad8baf admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:37.184709 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70ab87-3d2c-442d-8d00-7ce4a4ad8baf admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:37.190351 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c2e7c-3737-4dd3-a46d-4509c620b2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:37.190952 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c2e7c-3737-4dd3-a46d-4509c620b2ff admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:37.191285 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c2e7c-3737-4dd3-a46d-4509c620b2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:37.191558 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c2e7c-3737-4dd3-a46d-4509c620b2ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:37.192258 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c2e7c-3737-4dd3-a46d-4509c620b2ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:37.194010 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c2e7c-3737-4dd3-a46d-4509c620b2ff admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:37.204619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1453/2912] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:07:37.208722 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b60ad-4482-44fc-ad55-4d577df6112d admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:37.218837 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b60ad-4482-44fc-ad55-4d577df6112d admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:37.223985 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b15bdc-00f2-4910-87f3-a058d28a2303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:37.224588 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b15bdc-00f2-4910-87f3-a058d28a2303 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:37.224883 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b15bdc-00f2-4910-87f3-a058d28a2303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:37.225162 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b15bdc-00f2-4910-87f3-a058d28a2303 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:37.225798 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b15bdc-00f2-4910-87f3-a058d28a2303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:37.227432 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b15bdc-00f2-4910-87f3-a058d28a2303 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:37.228702 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:37.240421 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1460/2913] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:07:37.244627 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c2e7c-3737-4dd3-a46d-4509c620b2ff admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:37.255613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c2e7c-3737-4dd3-a46d-4509c620b2ff admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:37.260186 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8676d152-bbb8-4995-8ecc-b21e5602e637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:37.260829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676d152-bbb8-4995-8ecc-b21e5602e637 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:37.261112 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676d152-bbb8-4995-8ecc-b21e5602e637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:37.261388 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676d152-bbb8-4995-8ecc-b21e5602e637 admin admin] PATH_INFO: `/v3/projects/38170c47c95a47a8b133c153153fd4bb/users/1563ec29f6d54e159ab55746a0005a08/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:37.276680 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8676d152-bbb8-4995-8ecc-b21e5602e637 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38170c47c95a47a8b133c153153fd4bb, user_id=1563ec29f6d54e159ab55746a0005a08, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:37.279350 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8676d152-bbb8-4995-8ecc-b21e5602e637 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:37.293053 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1454/2914] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 01:07:37 2022] PUT /identity/v3/projects/38170c47c95a47a8b133c153153fd4bb/users/1563ec29f6d54e159ab55746a0005a08/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:37.303976 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a07ef-7f84-4ba3-bbb2-1468a97da1cd None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:37.304266 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a07ef-7f84-4ba3-bbb2-1468a97da1cd None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:37.304542 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a07ef-7f84-4ba3-bbb2-1468a97da1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:37.332386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-d39a07ef-7f84-4ba3-bbb2-1468a97da1cd None None] MFA Rules not processed for user `1563ec29f6d54e159ab55746a0005a08`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:37.333584 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-d39a07ef-7f84-4ba3-bbb2-1468a97da1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:37.399012 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1461/2915] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:07:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:07:37.403581 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8676d152-bbb8-4995-8ecc-b21e5602e637 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:37.435213 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:37.468988 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8676d152-bbb8-4995-8ecc-b21e5602e637 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:37.473921 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a19dc-90b8-4442-8718-63a058da91ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:37.474525 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a19dc-90b8-4442-8718-63a058da91ce admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:37.474802 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a19dc-90b8-4442-8718-63a058da91ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:37.475101 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a19dc-90b8-4442-8718-63a058da91ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:37.475728 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a19dc-90b8-4442-8718-63a058da91ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:37.477646 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a19dc-90b8-4442-8718-63a058da91ce admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:37.504824 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1455/2916] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 01:07:37 2022] POST /identity/v3/projects => generated 363 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:07:37.509112 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a07ef-7f84-4ba3-bbb2-1468a97da1cd None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:37.520169 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a07ef-7f84-4ba3-bbb2-1468a97da1cd None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:37.576173 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df1d9c-50da-4e13-a383-ff0654c27c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:37.577457 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df1d9c-50da-4e13-a383-ff0654c27c2f admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:37.577672 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df1d9c-50da-4e13-a383-ff0654c27c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:37.578049 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df1d9c-50da-4e13-a383-ff0654c27c2f admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:37.578697 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df1d9c-50da-4e13-a383-ff0654c27c2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:37.580642 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df1d9c-50da-4e13-a383-ff0654c27c2f admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:37.602363 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1462/2917] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 01:07:37 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:07:37.606365 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a19dc-90b8-4442-8718-63a058da91ce admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:37.617379 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a19dc-90b8-4442-8718-63a058da91ce admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:37.621969 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29137574-ca7c-4bf9-ac66-8d0d3332d89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:37.622562 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29137574-ca7c-4bf9-ac66-8d0d3332d89b admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:37.622843 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29137574-ca7c-4bf9-ac66-8d0d3332d89b admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:37.623153 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29137574-ca7c-4bf9-ac66-8d0d3332d89b admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:37.624439 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29137574-ca7c-4bf9-ac66-8d0d3332d89b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:37.626022 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29137574-ca7c-4bf9-ac66-8d0d3332d89b admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:37.636499 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1456/2918] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:07:37.641009 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df1d9c-50da-4e13-a383-ff0654c27c2f admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:37.651494 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df1d9c-50da-4e13-a383-ff0654c27c2f admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:37.656307 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5bacc2-6e29-4176-8fab-33c7b0759a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:37.656959 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bacc2-6e29-4176-8fab-33c7b0759a9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:37.657241 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bacc2-6e29-4176-8fab-33c7b0759a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:37.657518 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bacc2-6e29-4176-8fab-33c7b0759a9b admin admin] PATH_INFO: `/v3/projects/40c9e08f521c4a97ae0fa50b8f4c18e2/users/fd26667c89ea4f5bb89ecc0aa1aecdbe/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:37.672902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5bacc2-6e29-4176-8fab-33c7b0759a9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=40c9e08f521c4a97ae0fa50b8f4c18e2, user_id=fd26667c89ea4f5bb89ecc0aa1aecdbe, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:37.675609 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5bacc2-6e29-4176-8fab-33c7b0759a9b admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:37.691235 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1463/2919] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 01:07:37 2022] PUT /identity/v3/projects/40c9e08f521c4a97ae0fa50b8f4c18e2/users/fd26667c89ea4f5bb89ecc0aa1aecdbe/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:37.695253 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29137574-ca7c-4bf9-ac66-8d0d3332d89b admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:37.705568 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29137574-ca7c-4bf9-ac66-8d0d3332d89b admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:37.760703 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e441bc6-fce8-4ae0-bbb2-7e56de26b01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:37.761230 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e441bc6-fce8-4ae0-bbb2-7e56de26b01f admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:37.761509 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e441bc6-fce8-4ae0-bbb2-7e56de26b01f admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:37.761777 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e441bc6-fce8-4ae0-bbb2-7e56de26b01f admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:37.762405 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e441bc6-fce8-4ae0-bbb2-7e56de26b01f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:37.764116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e441bc6-fce8-4ae0-bbb2-7e56de26b01f admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:37.773599 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1457/2920] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:07:37.777720 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5bacc2-6e29-4176-8fab-33c7b0759a9b admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:37.788848 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5bacc2-6e29-4176-8fab-33c7b0759a9b admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:37.793607 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b1bea-da20-4229-a8d3-83e1e30e2bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:37.794254 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b1bea-da20-4229-a8d3-83e1e30e2bc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:37.794532 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b1bea-da20-4229-a8d3-83e1e30e2bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:37.794801 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b1bea-da20-4229-a8d3-83e1e30e2bc4 admin admin] PATH_INFO: `/v3/projects/40c9e08f521c4a97ae0fa50b8f4c18e2/users/fd26667c89ea4f5bb89ecc0aa1aecdbe/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:37.797323 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b1bea-da20-4229-a8d3-83e1e30e2bc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40c9e08f521c4a97ae0fa50b8f4c18e2, user_id=fd26667c89ea4f5bb89ecc0aa1aecdbe, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:37.800121 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b1bea-da20-4229-a8d3-83e1e30e2bc4 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:37.810460 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1464/2921] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 01:07:37 2022] PUT /identity/v3/projects/40c9e08f521c4a97ae0fa50b8f4c18e2/users/fd26667c89ea4f5bb89ecc0aa1aecdbe/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:37.820638 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622969d9-365c-479f-b10d-e39b018ed6a6 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:37.820924 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622969d9-365c-479f-b10d-e39b018ed6a6 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:37.821196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622969d9-365c-479f-b10d-e39b018ed6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:37.851536 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-622969d9-365c-479f-b10d-e39b018ed6a6 None None] MFA Rules not processed for user `fd26667c89ea4f5bb89ecc0aa1aecdbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:37.852752 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-622969d9-365c-479f-b10d-e39b018ed6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:37.921513 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1458/2922] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:07:37 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:07:37.932934 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b1bea-da20-4229-a8d3-83e1e30e2bc4 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:37.934684 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-845b1bea-da20-4229-a8d3-83e1e30e2bc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:37.978473 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:38.005230 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b1bea-da20-4229-a8d3-83e1e30e2bc4 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:38.009036 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fe915-b0e4-46f3-bce7-ee8ec156315d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:38.009641 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fe915-b0e4-46f3-bce7-ee8ec156315d service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:38.009950 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fe915-b0e4-46f3-bce7-ee8ec156315d service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:38.010221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fe915-b0e4-46f3-bce7-ee8ec156315d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:38.019714 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293fe915-b0e4-46f3-bce7-ee8ec156315d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:38.021091 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293fe915-b0e4-46f3-bce7-ee8ec156315d service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:38.035310 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1465/2923] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:07:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:07:38.112416 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622969d9-365c-479f-b10d-e39b018ed6a6 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:38.122093 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622969d9-365c-479f-b10d-e39b018ed6a6 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:38.125978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e050836-45d6-4974-9bab-f73a2d7030c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:38.126571 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e050836-45d6-4974-9bab-f73a2d7030c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:38.126849 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e050836-45d6-4974-9bab-f73a2d7030c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:38.127155 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e050836-45d6-4974-9bab-f73a2d7030c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:38.145700 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:38.189148 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e050836-45d6-4974-9bab-f73a2d7030c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:38.190588 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e050836-45d6-4974-9bab-f73a2d7030c5 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:38.203770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1459/2924] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:07:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:07:40.937815 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fe915-b0e4-46f3-bce7-ee8ec156315d service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:41.001171 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fe915-b0e4-46f3-bce7-ee8ec156315d service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:41.005802 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091392ac-b3e1-4548-9385-415398781aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:41.006435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091392ac-b3e1-4548-9385-415398781aa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:41.006716 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091392ac-b3e1-4548-9385-415398781aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:41.006993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091392ac-b3e1-4548-9385-415398781aa6 admin admin] PATH_INFO: `/v3/users/1563ec29f6d54e159ab55746a0005a08` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:41.008158 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091392ac-b3e1-4548-9385-415398781aa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1563ec29f6d54e159ab55746a0005a08)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:41.010083 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091392ac-b3e1-4548-9385-415398781aa6 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:41.054309 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-091392ac-b3e1-4548-9385-415398781aa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1563ec29f6d54e159ab55746a0005a08'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:41.062165 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-091392ac-b3e1-4548-9385-415398781aa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1563ec29f6d54e159ab55746a0005a08'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:41.068899 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-091392ac-b3e1-4548-9385-415398781aa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1563ec29f6d54e159ab55746a0005a08'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:41.069515 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-091392ac-b3e1-4548-9385-415398781aa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1563ec29f6d54e159ab55746a0005a08'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:41.070186 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-091392ac-b3e1-4548-9385-415398781aa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1563ec29f6d54e159ab55746a0005a08'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:41.087338 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1466/2925] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 01:07:40 2022] DELETE /identity/v3/users/1563ec29f6d54e159ab55746a0005a08 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:41.091815 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbb23c-78b3-4d09-89fa-1ce925c549c3 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:41.092494 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbb23c-78b3-4d09-89fa-1ce925c549c3 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:41.092494 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbb23c-78b3-4d09-89fa-1ce925c549c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:41.111520 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-12cbb23c-78b3-4d09-89fa-1ce925c549c3 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:41.112798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-12cbb23c-78b3-4d09-89fa-1ce925c549c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:41.170405 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1460/2926] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:07:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:07:41.180263 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091392ac-b3e1-4548-9385-415398781aa6 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:41.181666 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-091392ac-b3e1-4548-9385-415398781aa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:41.239243 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091392ac-b3e1-4548-9385-415398781aa6 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:41.242830 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6bd090-4587-4263-b4c0-6be4baaef0b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:41.243454 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bd090-4587-4263-b4c0-6be4baaef0b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:41.243732 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bd090-4587-4263-b4c0-6be4baaef0b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:41.244000 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bd090-4587-4263-b4c0-6be4baaef0b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:41.249686 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:41.255010 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6bd090-4587-4263-b4c0-6be4baaef0b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:41.257298 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6bd090-4587-4263-b4c0-6be4baaef0b9 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:41.267467 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1467/2927] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:07:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:07:41.537890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbb23c-78b3-4d09-89fa-1ce925c549c3 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:41.539484 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-12cbb23c-78b3-4d09-89fa-1ce925c549c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:41.552047 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbb23c-78b3-4d09-89fa-1ce925c549c3 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:41.556868 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21edf6df-3c68-4403-8759-c713befc0e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:41.557503 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edf6df-3c68-4403-8759-c713befc0e6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:41.557781 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edf6df-3c68-4403-8759-c713befc0e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:41.558053 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edf6df-3c68-4403-8759-c713befc0e6d admin admin] PATH_INFO: `/v3/projects/38170c47c95a47a8b133c153153fd4bb` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:41.559220 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21edf6df-3c68-4403-8759-c713befc0e6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=38170c47c95a47a8b133c153153fd4bb)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:41.561800 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21edf6df-3c68-4403-8759-c713befc0e6d admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:41.609831 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-21edf6df-3c68-4403-8759-c713befc0e6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38170c47c95a47a8b133c153153fd4bb', 'request_id': 'req-21edf6df-3c68-4403-8759-c713befc0e6d'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:41.617207 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-21edf6df-3c68-4403-8759-c713befc0e6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38170c47c95a47a8b133c153153fd4bb', 'request_id': 'req-21edf6df-3c68-4403-8759-c713befc0e6d'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:41.622515 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-21edf6df-3c68-4403-8759-c713befc0e6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38170c47c95a47a8b133c153153fd4bb', 'request_id': 'req-21edf6df-3c68-4403-8759-c713befc0e6d'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:41.634095 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-21edf6df-3c68-4403-8759-c713befc0e6d admin admin] The token cache is being invalidate because project 38170c47c95a47a8b133c153153fd4bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:07:41.634385 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-21edf6df-3c68-4403-8759-c713befc0e6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:41.635023 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-21edf6df-3c68-4403-8759-c713befc0e6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:41.636544 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1461/2928] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 01:07:41 2022] DELETE /identity/v3/projects/38170c47c95a47a8b133c153153fd4bb => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:41.640909 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6bd090-4587-4263-b4c0-6be4baaef0b9 service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:41.651290 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6bd090-4587-4263-b4c0-6be4baaef0b9 service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:41.707252 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc5e68-7545-4c1f-8b10-077f854829a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:41.707876 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc5e68-7545-4c1f-8b10-077f854829a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:41.708156 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc5e68-7545-4c1f-8b10-077f854829a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:41.708427 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc5e68-7545-4c1f-8b10-077f854829a4 admin admin] PATH_INFO: `/v3/users/fd26667c89ea4f5bb89ecc0aa1aecdbe` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:41.709595 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc5e68-7545-4c1f-8b10-077f854829a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd26667c89ea4f5bb89ecc0aa1aecdbe)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:41.711634 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc5e68-7545-4c1f-8b10-077f854829a4 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:41.771263 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-f8bc5e68-7545-4c1f-8b10-077f854829a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd26667c89ea4f5bb89ecc0aa1aecdbe'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:41.779204 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-f8bc5e68-7545-4c1f-8b10-077f854829a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd26667c89ea4f5bb89ecc0aa1aecdbe'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:41.798514 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-f8bc5e68-7545-4c1f-8b10-077f854829a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd26667c89ea4f5bb89ecc0aa1aecdbe'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:41.801346 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-f8bc5e68-7545-4c1f-8b10-077f854829a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd26667c89ea4f5bb89ecc0aa1aecdbe'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:41.802129 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-f8bc5e68-7545-4c1f-8b10-077f854829a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd26667c89ea4f5bb89ecc0aa1aecdbe'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:41.820314 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1468/2929] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 01:07:41 2022] DELETE /identity/v3/users/fd26667c89ea4f5bb89ecc0aa1aecdbe => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:42.090462 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21edf6df-3c68-4403-8759-c713befc0e6d admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:42.092026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-21edf6df-3c68-4403-8759-c713befc0e6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:42.107185 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21edf6df-3c68-4403-8759-c713befc0e6d admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:42.163552 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:42.179998 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b786fa20-7b22-4305-b590-81abcf6960f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:42.180548 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786fa20-7b22-4305-b590-81abcf6960f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:42.180822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786fa20-7b22-4305-b590-81abcf6960f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:42.181091 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786fa20-7b22-4305-b590-81abcf6960f0 admin admin] PATH_INFO: `/v3/projects/40c9e08f521c4a97ae0fa50b8f4c18e2` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:42.182218 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b786fa20-7b22-4305-b590-81abcf6960f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40c9e08f521c4a97ae0fa50b8f4c18e2)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:42.184136 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b786fa20-7b22-4305-b590-81abcf6960f0 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:42.230862 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b786fa20-7b22-4305-b590-81abcf6960f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40c9e08f521c4a97ae0fa50b8f4c18e2', 'request_id': 'req-b786fa20-7b22-4305-b590-81abcf6960f0'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:42.239585 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b786fa20-7b22-4305-b590-81abcf6960f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40c9e08f521c4a97ae0fa50b8f4c18e2', 'request_id': 'req-b786fa20-7b22-4305-b590-81abcf6960f0'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:42.246505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b786fa20-7b22-4305-b590-81abcf6960f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40c9e08f521c4a97ae0fa50b8f4c18e2', 'request_id': 'req-b786fa20-7b22-4305-b590-81abcf6960f0'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:42.258131 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b786fa20-7b22-4305-b590-81abcf6960f0 admin admin] The token cache is being invalidate because project 40c9e08f521c4a97ae0fa50b8f4c18e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:07:42.258424 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b786fa20-7b22-4305-b590-81abcf6960f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:42.259087 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-b786fa20-7b22-4305-b590-81abcf6960f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:42.260573 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1462/2930] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 01:07:42 2022] DELETE /identity/v3/projects/40c9e08f521c4a97ae0fa50b8f4c18e2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:42.266038 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c46b0-f155-4772-9092-3e6f95ee8214 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:42.266502 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c46b0-f155-4772-9092-3e6f95ee8214 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:42.266596 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c46b0-f155-4772-9092-3e6f95ee8214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:42.286644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-083c46b0-f155-4772-9092-3e6f95ee8214 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:42.287877 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-083c46b0-f155-4772-9092-3e6f95ee8214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:42.346941 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1469/2931] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:07:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:07:42.355640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72765c80-feae-4bda-98c7-381edc66ea04 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:42.355926 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72765c80-feae-4bda-98c7-381edc66ea04 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:42.356202 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72765c80-feae-4bda-98c7-381edc66ea04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:42.377144 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-72765c80-feae-4bda-98c7-381edc66ea04 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:42.378355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-72765c80-feae-4bda-98c7-381edc66ea04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:42.386337 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1463/2932] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:07:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:07:42.395904 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c46b0-f155-4772-9092-3e6f95ee8214 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:42.406431 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c46b0-f155-4772-9092-3e6f95ee8214 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:42.410953 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a44a7-adfb-41d7-a29c-88faf321eca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:42.411572 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a44a7-adfb-41d7-a29c-88faf321eca9 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:42.411851 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a44a7-adfb-41d7-a29c-88faf321eca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:42.412122 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a44a7-adfb-41d7-a29c-88faf321eca9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:42.412763 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a44a7-adfb-41d7-a29c-88faf321eca9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:42.413065 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a44a7-adfb-41d7-a29c-88faf321eca9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:42.414667 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a44a7-adfb-41d7-a29c-88faf321eca9 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:42.423837 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:42.428589 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1470/2933] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 01:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:42.431913 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72765c80-feae-4bda-98c7-381edc66ea04 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:42.442299 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72765c80-feae-4bda-98c7-381edc66ea04 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:42.446970 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb5fee-aa15-4f78-aa0c-7d72497d71a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:42.447598 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb5fee-aa15-4f78-aa0c-7d72497d71a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:42.447878 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb5fee-aa15-4f78-aa0c-7d72497d71a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:42.448147 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb5fee-aa15-4f78-aa0c-7d72497d71a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:42.448773 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbb5fee-aa15-4f78-aa0c-7d72497d71a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:42.450673 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbb5fee-aa15-4f78-aa0c-7d72497d71a6 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:42.479481 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1464/2934] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 01:07:42 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:07:42.484274 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a44a7-adfb-41d7-a29c-88faf321eca9 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:42.494238 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a44a7-adfb-41d7-a29c-88faf321eca9 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:42.550177 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab581e-26ee-43e3-918b-73a840a57da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:42.550782 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab581e-26ee-43e3-918b-73a840a57da4 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:42.551195 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab581e-26ee-43e3-918b-73a840a57da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:42.551377 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab581e-26ee-43e3-918b-73a840a57da4 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:42.551997 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab581e-26ee-43e3-918b-73a840a57da4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:42.553953 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab581e-26ee-43e3-918b-73a840a57da4 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:42.575446 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1471/2935] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 01:07:42 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:07:42.580275 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb5fee-aa15-4f78-aa0c-7d72497d71a6 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:42.590588 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb5fee-aa15-4f78-aa0c-7d72497d71a6 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:42.595618 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dcde98-4a21-49d4-a686-b5027d1e6f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:42.596571 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dcde98-4a21-49d4-a686-b5027d1e6f1e admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:42.596948 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dcde98-4a21-49d4-a686-b5027d1e6f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:42.597312 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dcde98-4a21-49d4-a686-b5027d1e6f1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:42.598232 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dcde98-4a21-49d4-a686-b5027d1e6f1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:42.600152 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dcde98-4a21-49d4-a686-b5027d1e6f1e admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:42.610566 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1465/2936] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:07:42.614571 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab581e-26ee-43e3-918b-73a840a57da4 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:42.625489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab581e-26ee-43e3-918b-73a840a57da4 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:42.630154 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed038e-15be-48e2-9cb5-b54384983dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:42.630753 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed038e-15be-48e2-9cb5-b54384983dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:42.631048 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed038e-15be-48e2-9cb5-b54384983dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:42.631335 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed038e-15be-48e2-9cb5-b54384983dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:42.631974 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed038e-15be-48e2-9cb5-b54384983dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:42.633567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed038e-15be-48e2-9cb5-b54384983dd1 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:42.645206 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1472/2937] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:07:42.649496 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dcde98-4a21-49d4-a686-b5027d1e6f1e admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:42.660122 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dcde98-4a21-49d4-a686-b5027d1e6f1e admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:42.664784 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbd556-c66d-49ce-9303-68869afebb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:42.665456 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbd556-c66d-49ce-9303-68869afebb5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:42.665735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbd556-c66d-49ce-9303-68869afebb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:42.666009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbd556-c66d-49ce-9303-68869afebb5e admin admin] PATH_INFO: `/v3/projects/5201fabd1f3145428269631bddf347e6/users/24c26ec9497c4a018c706ade6abd45d5/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:42.680495 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdbd556-c66d-49ce-9303-68869afebb5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5201fabd1f3145428269631bddf347e6, user_id=24c26ec9497c4a018c706ade6abd45d5, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:42.683233 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdbd556-c66d-49ce-9303-68869afebb5e admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:42.696349 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1466/2938] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 01:07:42 2022] PUT /identity/v3/projects/5201fabd1f3145428269631bddf347e6/users/24c26ec9497c4a018c706ade6abd45d5/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:42.707228 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ddf4a-d30d-48a4-9d4e-401a399f95d9 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:42.707541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ddf4a-d30d-48a4-9d4e-401a399f95d9 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:42.707810 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ddf4a-d30d-48a4-9d4e-401a399f95d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:42.735582 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-133ddf4a-d30d-48a4-9d4e-401a399f95d9 None None] MFA Rules not processed for user `24c26ec9497c4a018c706ade6abd45d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:42.736836 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-133ddf4a-d30d-48a4-9d4e-401a399f95d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:42.799991 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1473/2939] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:07:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:07:42.804505 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbd556-c66d-49ce-9303-68869afebb5e admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:42.866850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbd556-c66d-49ce-9303-68869afebb5e admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:42.871535 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebe6fe-649f-44bb-b0e4-4c21767aa4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:42.872145 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebe6fe-649f-44bb-b0e4-4c21767aa4e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:42.872426 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebe6fe-649f-44bb-b0e4-4c21767aa4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:42.872698 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebe6fe-649f-44bb-b0e4-4c21767aa4e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:42.873910 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ebe6fe-649f-44bb-b0e4-4c21767aa4e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:42.875838 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ebe6fe-649f-44bb-b0e4-4c21767aa4e1 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:42.891654 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:42.905282 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1467/2940] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 01:07:42 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:07:42.909579 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ddf4a-d30d-48a4-9d4e-401a399f95d9 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:42.920349 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ddf4a-d30d-48a4-9d4e-401a399f95d9 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:42.976347 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0651ac-726e-4c8f-ac79-e0d4619ae825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:42.976910 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0651ac-726e-4c8f-ac79-e0d4619ae825 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:42.977189 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0651ac-726e-4c8f-ac79-e0d4619ae825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:42.977455 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0651ac-726e-4c8f-ac79-e0d4619ae825 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:42.978104 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0651ac-726e-4c8f-ac79-e0d4619ae825 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:42.980054 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0651ac-726e-4c8f-ac79-e0d4619ae825 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:42.995755 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:43.003170 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1474/2941] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 01:07:42 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:07:43.006819 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebe6fe-649f-44bb-b0e4-4c21767aa4e1 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:43.017962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebe6fe-649f-44bb-b0e4-4c21767aa4e1 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:43.022684 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c782a-964b-47ab-93c5-9fc106974cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:43.023317 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c782a-964b-47ab-93c5-9fc106974cb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:43.023600 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c782a-964b-47ab-93c5-9fc106974cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:43.023873 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c782a-964b-47ab-93c5-9fc106974cb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:43.024515 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c782a-964b-47ab-93c5-9fc106974cb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:43.026101 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c782a-964b-47ab-93c5-9fc106974cb6 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:43.038839 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1468/2942] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:07:43.042838 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0651ac-726e-4c8f-ac79-e0d4619ae825 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:43.053031 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0651ac-726e-4c8f-ac79-e0d4619ae825 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:43.057643 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20503c6d-1aa9-4dc2-a281-f7c76c4bbb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:43.058240 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20503c6d-1aa9-4dc2-a281-f7c76c4bbb3c admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:43.058543 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20503c6d-1aa9-4dc2-a281-f7c76c4bbb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:43.058815 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20503c6d-1aa9-4dc2-a281-f7c76c4bbb3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:43.059561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20503c6d-1aa9-4dc2-a281-f7c76c4bbb3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:43.061213 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20503c6d-1aa9-4dc2-a281-f7c76c4bbb3c admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:43.070796 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1475/2943] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:07:43.074840 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c782a-964b-47ab-93c5-9fc106974cb6 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:43.085369 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c782a-964b-47ab-93c5-9fc106974cb6 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:43.089994 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d0100-b5bb-450b-82fb-850f5159b821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:43.090643 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d0100-b5bb-450b-82fb-850f5159b821 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:43.090922 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d0100-b5bb-450b-82fb-850f5159b821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:43.091231 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d0100-b5bb-450b-82fb-850f5159b821 admin admin] PATH_INFO: `/v3/projects/499159f42de64f898a6a92a617d6cb60/users/9e0cef4e7ded42eaae459605b4e5bc69/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:43.106599 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758d0100-b5bb-450b-82fb-850f5159b821 admin admin] RBAC: Authorizing `identity:create_grant(project_id=499159f42de64f898a6a92a617d6cb60, user_id=9e0cef4e7ded42eaae459605b4e5bc69, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:43.109304 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758d0100-b5bb-450b-82fb-850f5159b821 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:43.121862 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1469/2944] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 01:07:43 2022] PUT /identity/v3/projects/499159f42de64f898a6a92a617d6cb60/users/9e0cef4e7ded42eaae459605b4e5bc69/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:43.133023 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8a78c-77b2-4709-8dc2-85f77fa9224f None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:43.133313 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8a78c-77b2-4709-8dc2-85f77fa9224f None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:43.133604 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8a78c-77b2-4709-8dc2-85f77fa9224f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:43.173019 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-34e8a78c-77b2-4709-8dc2-85f77fa9224f None None] MFA Rules not processed for user `9e0cef4e7ded42eaae459605b4e5bc69`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:43.174230 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-34e8a78c-77b2-4709-8dc2-85f77fa9224f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:43.238111 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1476/2945] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:07:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:07:43.242465 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d0100-b5bb-450b-82fb-850f5159b821 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:43.303317 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d0100-b5bb-450b-82fb-850f5159b821 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:43.308013 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80688b85-4746-4f10-80cd-1685dfd4688f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:43.308613 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80688b85-4746-4f10-80cd-1685dfd4688f admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:43.308892 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80688b85-4746-4f10-80cd-1685dfd4688f admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:43.309163 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80688b85-4746-4f10-80cd-1685dfd4688f admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:43.309822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80688b85-4746-4f10-80cd-1685dfd4688f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:43.312349 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80688b85-4746-4f10-80cd-1685dfd4688f admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:43.338897 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1470/2946] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 01:07:43 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:07:43.343379 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8a78c-77b2-4709-8dc2-85f77fa9224f None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:43.354565 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8a78c-77b2-4709-8dc2-85f77fa9224f None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:43.411881 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23150f2-1a4e-40c7-914c-a67645466061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:43.412482 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23150f2-1a4e-40c7-914c-a67645466061 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:43.412758 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23150f2-1a4e-40c7-914c-a67645466061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:43.413026 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23150f2-1a4e-40c7-914c-a67645466061 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:43.413686 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23150f2-1a4e-40c7-914c-a67645466061 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:43.415622 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23150f2-1a4e-40c7-914c-a67645466061 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:43.436930 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1477/2947] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 01:07:43 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 01:07:43.441121 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80688b85-4746-4f10-80cd-1685dfd4688f admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:43.451814 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80688b85-4746-4f10-80cd-1685dfd4688f admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:43.456745 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a5b78-978f-4030-a9cf-3b6707dedc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:43.457355 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a5b78-978f-4030-a9cf-3b6707dedc2b admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:43.457636 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a5b78-978f-4030-a9cf-3b6707dedc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:43.457910 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a5b78-978f-4030-a9cf-3b6707dedc2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:43.458553 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a5b78-978f-4030-a9cf-3b6707dedc2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:43.460173 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a5b78-978f-4030-a9cf-3b6707dedc2b admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:43.481085 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1471/2948] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:07:43.486343 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23150f2-1a4e-40c7-914c-a67645466061 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:43.497596 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23150f2-1a4e-40c7-914c-a67645466061 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:43.503585 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ef73e-7233-42dd-82d1-7a77bc454346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:43.504261 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ef73e-7233-42dd-82d1-7a77bc454346 admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:43.504541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ef73e-7233-42dd-82d1-7a77bc454346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:43.504816 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ef73e-7233-42dd-82d1-7a77bc454346 admin admin] PATH_INFO: `/v3/projects/d663358b2446435baba8fa1326404814/users/35cf27264f8a4f6e8b3ef5c9b6b95fc2/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:43.522393 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ef73e-7233-42dd-82d1-7a77bc454346 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d663358b2446435baba8fa1326404814, user_id=35cf27264f8a4f6e8b3ef5c9b6b95fc2, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:43.525115 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ef73e-7233-42dd-82d1-7a77bc454346 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:43.541308 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1478/2949] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 01:07:43 2022] PUT /identity/v3/projects/d663358b2446435baba8fa1326404814/users/35cf27264f8a4f6e8b3ef5c9b6b95fc2/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:43.546794 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a5b78-978f-4030-a9cf-3b6707dedc2b admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:43.560567 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a5b78-978f-4030-a9cf-3b6707dedc2b admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:43.616962 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417746dc-5ce6-4aa5-87ea-adba8cf40716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:43.617566 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417746dc-5ce6-4aa5-87ea-adba8cf40716 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:43.617847 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417746dc-5ce6-4aa5-87ea-adba8cf40716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:43.618118 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417746dc-5ce6-4aa5-87ea-adba8cf40716 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:43.618772 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417746dc-5ce6-4aa5-87ea-adba8cf40716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:43.620374 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417746dc-5ce6-4aa5-87ea-adba8cf40716 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:43.630129 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1472/2950] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:07:43.634360 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ef73e-7233-42dd-82d1-7a77bc454346 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:43.645767 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ef73e-7233-42dd-82d1-7a77bc454346 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:43.651424 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2483f-ebe9-4e68-a9d4-88688a1c9d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:43.652113 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2483f-ebe9-4e68-a9d4-88688a1c9d2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:43.652397 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2483f-ebe9-4e68-a9d4-88688a1c9d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:43.652672 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2483f-ebe9-4e68-a9d4-88688a1c9d2b admin admin] PATH_INFO: `/v3/projects/d663358b2446435baba8fa1326404814/users/35cf27264f8a4f6e8b3ef5c9b6b95fc2/roles/34df06ce84ad4cf8b3bd1a4490228017` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:43.655485 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2483f-ebe9-4e68-a9d4-88688a1c9d2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d663358b2446435baba8fa1326404814, user_id=35cf27264f8a4f6e8b3ef5c9b6b95fc2, role_id=34df06ce84ad4cf8b3bd1a4490228017)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:43.658361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2483f-ebe9-4e68-a9d4-88688a1c9d2b admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:43.668935 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1479/2951] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 01:07:43 2022] PUT /identity/v3/projects/d663358b2446435baba8fa1326404814/users/35cf27264f8a4f6e8b3ef5c9b6b95fc2/roles/34df06ce84ad4cf8b3bd1a4490228017 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:43.679301 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424767c-c71b-4385-b2ba-a44d06579405 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:43.679589 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424767c-c71b-4385-b2ba-a44d06579405 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:43.679863 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424767c-c71b-4385-b2ba-a44d06579405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:43.708713 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-7424767c-c71b-4385-b2ba-a44d06579405 None None] MFA Rules not processed for user `35cf27264f8a4f6e8b3ef5c9b6b95fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:43.709927 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-7424767c-c71b-4385-b2ba-a44d06579405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:43.728943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:43.780563 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1473/2952] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:07:43 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:07:43.792092 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2483f-ebe9-4e68-a9d4-88688a1c9d2b admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:43.793807 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-bdc2483f-ebe9-4e68-a9d4-88688a1c9d2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:43.829268 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:43.864058 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2483f-ebe9-4e68-a9d4-88688a1c9d2b admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:43.867837 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4edc9c-0074-46c3-84ee-ddc69e3bef48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:43.868452 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4edc9c-0074-46c3-84ee-ddc69e3bef48 service cinder] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:43.868732 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4edc9c-0074-46c3-84ee-ddc69e3bef48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:43.869005 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4edc9c-0074-46c3-84ee-ddc69e3bef48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:43.882922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4edc9c-0074-46c3-84ee-ddc69e3bef48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:43.884406 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4edc9c-0074-46c3-84ee-ddc69e3bef48 service cinder] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:43.897440 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1480/2953] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:07:43 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:07:43.939756 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424767c-c71b-4385-b2ba-a44d06579405 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:43.941159 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-7424767c-c71b-4385-b2ba-a44d06579405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:43.953433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424767c-c71b-4385-b2ba-a44d06579405 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:43.956994 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb9d947-b00f-41ec-ab0d-d697b0c4f9d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:43.957581 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb9d947-b00f-41ec-ab0d-d697b0c4f9d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:43.957855 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb9d947-b00f-41ec-ab0d-d697b0c4f9d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:43.958120 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb9d947-b00f-41ec-ab0d-d697b0c4f9d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:44.017152 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb9d947-b00f-41ec-ab0d-d697b0c4f9d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:44.018780 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb9d947-b00f-41ec-ab0d-d697b0c4f9d5 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:44.027852 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1474/2954] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:07:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:07:46.926623 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4edc9c-0074-46c3-84ee-ddc69e3bef48 service cinder] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:46.987142 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4edc9c-0074-46c3-84ee-ddc69e3bef48 service cinder] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:46.991690 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6deb9-d9f9-4f0f-8988-dddde351ea1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:46.992312 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6deb9-d9f9-4f0f-8988-dddde351ea1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:46.992588 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6deb9-d9f9-4f0f-8988-dddde351ea1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:46.992854 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6deb9-d9f9-4f0f-8988-dddde351ea1c admin admin] PATH_INFO: `/v3/users/24c26ec9497c4a018c706ade6abd45d5` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:46.994013 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6deb9-d9f9-4f0f-8988-dddde351ea1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=24c26ec9497c4a018c706ade6abd45d5)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:46.995950 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6deb9-d9f9-4f0f-8988-dddde351ea1c admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:47.039576 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-4eb6deb9-d9f9-4f0f-8988-dddde351ea1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24c26ec9497c4a018c706ade6abd45d5'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:47.048162 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-4eb6deb9-d9f9-4f0f-8988-dddde351ea1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24c26ec9497c4a018c706ade6abd45d5'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:47.055106 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-4eb6deb9-d9f9-4f0f-8988-dddde351ea1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24c26ec9497c4a018c706ade6abd45d5'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:47.055728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-4eb6deb9-d9f9-4f0f-8988-dddde351ea1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24c26ec9497c4a018c706ade6abd45d5'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:47.056322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-4eb6deb9-d9f9-4f0f-8988-dddde351ea1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24c26ec9497c4a018c706ade6abd45d5'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:47.073419 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1481/2955] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 01:07:46 2022] DELETE /identity/v3/users/24c26ec9497c4a018c706ade6abd45d5 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:47.078311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86665456-582d-4e5b-aae5-a5f98c6143c0 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:47.078769 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86665456-582d-4e5b-aae5-a5f98c6143c0 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:47.078914 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86665456-582d-4e5b-aae5-a5f98c6143c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:47.099401 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-86665456-582d-4e5b-aae5-a5f98c6143c0 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:47.100694 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-86665456-582d-4e5b-aae5-a5f98c6143c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:47.160797 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1475/2956] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:07:47.170600 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6deb9-d9f9-4f0f-8988-dddde351ea1c admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:47.172032 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4eb6deb9-d9f9-4f0f-8988-dddde351ea1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:47.229477 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6deb9-d9f9-4f0f-8988-dddde351ea1c admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:47.233369 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b1fde-e283-426c-a778-b57ef15481b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:47.233976 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b1fde-e283-426c-a778-b57ef15481b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:47.234285 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b1fde-e283-426c-a778-b57ef15481b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:47.234558 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b1fde-e283-426c-a778-b57ef15481b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:47.240407 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:47.244912 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1b1fde-e283-426c-a778-b57ef15481b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:47.247124 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1b1fde-e283-426c-a778-b57ef15481b4 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:47.257860 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1482/2957] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:07:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:07:47.535152 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86665456-582d-4e5b-aae5-a5f98c6143c0 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:47.536594 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-86665456-582d-4e5b-aae5-a5f98c6143c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:47.548756 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86665456-582d-4e5b-aae5-a5f98c6143c0 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:47.553389 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb41928-34e8-4389-9ce4-0fc539965bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:47.554024 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb41928-34e8-4389-9ce4-0fc539965bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:47.554314 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb41928-34e8-4389-9ce4-0fc539965bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:47.554591 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb41928-34e8-4389-9ce4-0fc539965bac admin admin] PATH_INFO: `/v3/projects/5201fabd1f3145428269631bddf347e6` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:47.555735 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb41928-34e8-4389-9ce4-0fc539965bac admin admin] RBAC: Authorizing `identity:delete_project(project_id=5201fabd1f3145428269631bddf347e6)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:47.557649 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb41928-34e8-4389-9ce4-0fc539965bac admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:47.606457 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-6bb41928-34e8-4389-9ce4-0fc539965bac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5201fabd1f3145428269631bddf347e6', 'request_id': 'req-6bb41928-34e8-4389-9ce4-0fc539965bac'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:47.608205 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:47.613803 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-6bb41928-34e8-4389-9ce4-0fc539965bac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5201fabd1f3145428269631bddf347e6', 'request_id': 'req-6bb41928-34e8-4389-9ce4-0fc539965bac'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:47.619684 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-6bb41928-34e8-4389-9ce4-0fc539965bac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5201fabd1f3145428269631bddf347e6', 'request_id': 'req-6bb41928-34e8-4389-9ce4-0fc539965bac'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:47.630749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-6bb41928-34e8-4389-9ce4-0fc539965bac admin admin] The token cache is being invalidate because project 5201fabd1f3145428269631bddf347e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:07:47.631061 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-6bb41928-34e8-4389-9ce4-0fc539965bac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:47.631703 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-6bb41928-34e8-4389-9ce4-0fc539965bac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:47.633189 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1476/2958] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 01:07:47 2022] DELETE /identity/v3/projects/5201fabd1f3145428269631bddf347e6 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:47.637189 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b1fde-e283-426c-a778-b57ef15481b4 service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:47.647758 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b1fde-e283-426c-a778-b57ef15481b4 service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:47.705489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ca59b-0aaa-4f1c-a7ce-00d553786c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:47.706178 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ca59b-0aaa-4f1c-a7ce-00d553786c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:47.706385 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ca59b-0aaa-4f1c-a7ce-00d553786c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:47.706691 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ca59b-0aaa-4f1c-a7ce-00d553786c30 admin admin] PATH_INFO: `/v3/users/9e0cef4e7ded42eaae459605b4e5bc69` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:47.707851 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ca59b-0aaa-4f1c-a7ce-00d553786c30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e0cef4e7ded42eaae459605b4e5bc69)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:47.709766 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ca59b-0aaa-4f1c-a7ce-00d553786c30 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:47.753646 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-7f4ca59b-0aaa-4f1c-a7ce-00d553786c30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e0cef4e7ded42eaae459605b4e5bc69'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:47.760548 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-7f4ca59b-0aaa-4f1c-a7ce-00d553786c30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e0cef4e7ded42eaae459605b4e5bc69'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:47.767828 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-7f4ca59b-0aaa-4f1c-a7ce-00d553786c30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e0cef4e7ded42eaae459605b4e5bc69'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:47.768478 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-7f4ca59b-0aaa-4f1c-a7ce-00d553786c30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e0cef4e7ded42eaae459605b4e5bc69'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:47.769089 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-7f4ca59b-0aaa-4f1c-a7ce-00d553786c30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e0cef4e7ded42eaae459605b4e5bc69'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:47.786431 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1483/2959] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 01:07:47 2022] DELETE /identity/v3/users/9e0cef4e7ded42eaae459605b4e5bc69 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:48.059222 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb41928-34e8-4389-9ce4-0fc539965bac admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:48.061648 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-6bb41928-34e8-4389-9ce4-0fc539965bac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:48.077300 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb41928-34e8-4389-9ce4-0fc539965bac admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:48.137671 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32bc34a-e2dc-4572-94f0-189be8288cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:48.138322 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bc34a-e2dc-4572-94f0-189be8288cd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:48.138483 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bc34a-e2dc-4572-94f0-189be8288cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:48.138870 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bc34a-e2dc-4572-94f0-189be8288cd0 admin admin] PATH_INFO: `/v3/projects/499159f42de64f898a6a92a617d6cb60` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:48.140380 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32bc34a-e2dc-4572-94f0-189be8288cd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=499159f42de64f898a6a92a617d6cb60)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:48.142295 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32bc34a-e2dc-4572-94f0-189be8288cd0 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:48.189961 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-a32bc34a-e2dc-4572-94f0-189be8288cd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '499159f42de64f898a6a92a617d6cb60', 'request_id': 'req-a32bc34a-e2dc-4572-94f0-189be8288cd0'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:48.197173 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-a32bc34a-e2dc-4572-94f0-189be8288cd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '499159f42de64f898a6a92a617d6cb60', 'request_id': 'req-a32bc34a-e2dc-4572-94f0-189be8288cd0'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:48.203377 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-a32bc34a-e2dc-4572-94f0-189be8288cd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '499159f42de64f898a6a92a617d6cb60', 'request_id': 'req-a32bc34a-e2dc-4572-94f0-189be8288cd0'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:48.215121 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-a32bc34a-e2dc-4572-94f0-189be8288cd0 admin admin] The token cache is being invalidate because project 499159f42de64f898a6a92a617d6cb60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:07:48.215411 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-a32bc34a-e2dc-4572-94f0-189be8288cd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:48.216080 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-a32bc34a-e2dc-4572-94f0-189be8288cd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:48.217897 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1477/2960] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 01:07:48 2022] DELETE /identity/v3/projects/499159f42de64f898a6a92a617d6cb60 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:48.222325 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ca59b-0aaa-4f1c-a7ce-00d553786c30 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:48.223799 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-7f4ca59b-0aaa-4f1c-a7ce-00d553786c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:48.237367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ca59b-0aaa-4f1c-a7ce-00d553786c30 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:48.292280 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d65469-0b2c-4624-8a98-d5d740a1c452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:48.292906 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d65469-0b2c-4624-8a98-d5d740a1c452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:48.293181 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d65469-0b2c-4624-8a98-d5d740a1c452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:48.293452 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d65469-0b2c-4624-8a98-d5d740a1c452 admin admin] PATH_INFO: `/v3/users/35cf27264f8a4f6e8b3ef5c9b6b95fc2` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:48.294555 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d65469-0b2c-4624-8a98-d5d740a1c452 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35cf27264f8a4f6e8b3ef5c9b6b95fc2)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:48.297006 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d65469-0b2c-4624-8a98-d5d740a1c452 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:48.310090 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:48.346831 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-54d65469-0b2c-4624-8a98-d5d740a1c452 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35cf27264f8a4f6e8b3ef5c9b6b95fc2'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:48.353767 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-54d65469-0b2c-4624-8a98-d5d740a1c452 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35cf27264f8a4f6e8b3ef5c9b6b95fc2'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:48.361061 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-54d65469-0b2c-4624-8a98-d5d740a1c452 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35cf27264f8a4f6e8b3ef5c9b6b95fc2'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:48.361701 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-54d65469-0b2c-4624-8a98-d5d740a1c452 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35cf27264f8a4f6e8b3ef5c9b6b95fc2'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:48.362334 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-54d65469-0b2c-4624-8a98-d5d740a1c452 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35cf27264f8a4f6e8b3ef5c9b6b95fc2'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:48.379618 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1484/2961] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 01:07:48 2022] DELETE /identity/v3/users/35cf27264f8a4f6e8b3ef5c9b6b95fc2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 01:07:48.664359 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32bc34a-e2dc-4572-94f0-189be8288cd0 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:48.665806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-a32bc34a-e2dc-4572-94f0-189be8288cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:48.680423 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32bc34a-e2dc-4572-94f0-189be8288cd0 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:48.739483 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc4190c-c238-4ceb-bd32-9fe3b582d42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:48.740119 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4190c-c238-4ceb-bd32-9fe3b582d42d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:48.740401 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4190c-c238-4ceb-bd32-9fe3b582d42d admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:48.740674 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4190c-c238-4ceb-bd32-9fe3b582d42d admin admin] PATH_INFO: `/v3/projects/d663358b2446435baba8fa1326404814` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:48.741794 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc4190c-c238-4ceb-bd32-9fe3b582d42d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d663358b2446435baba8fa1326404814)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:48.743758 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc4190c-c238-4ceb-bd32-9fe3b582d42d admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:48.754580 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:48.790929 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-3dc4190c-c238-4ceb-bd32-9fe3b582d42d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd663358b2446435baba8fa1326404814', 'request_id': 'req-3dc4190c-c238-4ceb-bd32-9fe3b582d42d'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:48.798978 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-3dc4190c-c238-4ceb-bd32-9fe3b582d42d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd663358b2446435baba8fa1326404814', 'request_id': 'req-3dc4190c-c238-4ceb-bd32-9fe3b582d42d'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:48.804664 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-3dc4190c-c238-4ceb-bd32-9fe3b582d42d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd663358b2446435baba8fa1326404814', 'request_id': 'req-3dc4190c-c238-4ceb-bd32-9fe3b582d42d'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:48.816042 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-3dc4190c-c238-4ceb-bd32-9fe3b582d42d admin admin] The token cache is being invalidate because project d663358b2446435baba8fa1326404814 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:07:48.816329 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-3dc4190c-c238-4ceb-bd32-9fe3b582d42d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:48.816961 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-3dc4190c-c238-4ceb-bd32-9fe3b582d42d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:48.818413 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1478/2962] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 01:07:48 2022] DELETE /identity/v3/projects/d663358b2446435baba8fa1326404814 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:48.824363 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553c109-9450-4848-96c8-e5099cf97b0d None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:48.824657 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553c109-9450-4848-96c8-e5099cf97b0d None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:48.824933 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553c109-9450-4848-96c8-e5099cf97b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:48.845312 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-8553c109-9450-4848-96c8-e5099cf97b0d None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:48.846786 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-8553c109-9450-4848-96c8-e5099cf97b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:48.906532 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1485/2963] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:07:48.915069 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c6f51-cead-4149-b802-e93c13c73b73 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:48.915306 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c6f51-cead-4149-b802-e93c13c73b73 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:48.915579 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c6f51-cead-4149-b802-e93c13c73b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:48.936897 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-ff8c6f51-cead-4149-b802-e93c13c73b73 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:48.938119 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-ff8c6f51-cead-4149-b802-e93c13c73b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:48.946111 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1479/2964] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:07:48.950507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553c109-9450-4848-96c8-e5099cf97b0d None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:48.960864 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553c109-9450-4848-96c8-e5099cf97b0d None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:48.965485 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00d612-d94c-4605-9196-1b05b4829f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:48.966082 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00d612-d94c-4605-9196-1b05b4829f63 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:48.966362 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00d612-d94c-4605-9196-1b05b4829f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:48.966635 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00d612-d94c-4605-9196-1b05b4829f63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:48.967644 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00d612-d94c-4605-9196-1b05b4829f63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:48.967953 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00d612-d94c-4605-9196-1b05b4829f63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:48.969567 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00d612-d94c-4605-9196-1b05b4829f63 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:48.981428 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1486/2965] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 01:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:07:48.986471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c6f51-cead-4149-b802-e93c13c73b73 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:48.996009 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c6f51-cead-4149-b802-e93c13c73b73 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:49.001867 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb099bb-b6db-41cb-92be-1cedd4d0fdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:49.002482 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb099bb-b6db-41cb-92be-1cedd4d0fdcd admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:49.002760 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb099bb-b6db-41cb-92be-1cedd4d0fdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:49.003030 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb099bb-b6db-41cb-92be-1cedd4d0fdcd admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:49.003698 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb099bb-b6db-41cb-92be-1cedd4d0fdcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:49.005595 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb099bb-b6db-41cb-92be-1cedd4d0fdcd admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:49.034103 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1480/2966] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 01:07:48 2022] POST /identity/v3/projects => generated 361 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:07:49.038250 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00d612-d94c-4605-9196-1b05b4829f63 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:49.048953 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00d612-d94c-4605-9196-1b05b4829f63 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:49.104169 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91734dec-0e2c-4b99-abe7-ee1c16942bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:49.104773 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91734dec-0e2c-4b99-abe7-ee1c16942bc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:49.105052 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91734dec-0e2c-4b99-abe7-ee1c16942bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:49.105320 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91734dec-0e2c-4b99-abe7-ee1c16942bc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:49.105945 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91734dec-0e2c-4b99-abe7-ee1c16942bc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:49.108426 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91734dec-0e2c-4b99-abe7-ee1c16942bc1 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:49.130351 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1487/2967] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 01:07:49 2022] POST /identity/v3/users => generated 281 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:07:49.136019 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb099bb-b6db-41cb-92be-1cedd4d0fdcd admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:49.147022 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb099bb-b6db-41cb-92be-1cedd4d0fdcd admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:49.152112 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695f259-0628-46c7-a796-54e22982a8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:49.152770 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695f259-0628-46c7-a796-54e22982a8b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:49.153112 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695f259-0628-46c7-a796-54e22982a8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:49.153451 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695f259-0628-46c7-a796-54e22982a8b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:49.154192 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4695f259-0628-46c7-a796-54e22982a8b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:49.156706 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4695f259-0628-46c7-a796-54e22982a8b7 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:49.166526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1481/2968] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:07:49.170649 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91734dec-0e2c-4b99-abe7-ee1c16942bc1 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:49.180926 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91734dec-0e2c-4b99-abe7-ee1c16942bc1 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:49.186224 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a2a0c-4689-407b-aec9-f27bb1c22252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:49.187132 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a2a0c-4689-407b-aec9-f27bb1c22252 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:49.187422 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a2a0c-4689-407b-aec9-f27bb1c22252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:49.187726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a2a0c-4689-407b-aec9-f27bb1c22252 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:49.188367 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a2a0c-4689-407b-aec9-f27bb1c22252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:49.189967 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a2a0c-4689-407b-aec9-f27bb1c22252 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:49.192046 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:49.203561 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1488/2969] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:07:49.207729 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695f259-0628-46c7-a796-54e22982a8b7 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:49.218253 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695f259-0628-46c7-a796-54e22982a8b7 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:49.223260 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefe398-cfaf-4ce1-be0a-d0e0d33b5dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:49.223913 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefe398-cfaf-4ce1-be0a-d0e0d33b5dc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:49.224196 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefe398-cfaf-4ce1-be0a-d0e0d33b5dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:49.224471 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefe398-cfaf-4ce1-be0a-d0e0d33b5dc0 admin admin] PATH_INFO: `/v3/projects/a4b6f9b1e9cd4c4c9007c8c006427bee/users/db9b6a7144e04fb0ac4955b61ed0cacd/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:49.239290 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcefe398-cfaf-4ce1-be0a-d0e0d33b5dc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4b6f9b1e9cd4c4c9007c8c006427bee, user_id=db9b6a7144e04fb0ac4955b61ed0cacd, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:49.241938 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcefe398-cfaf-4ce1-be0a-d0e0d33b5dc0 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:49.256761 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1482/2970] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 01:07:49 2022] PUT /identity/v3/projects/a4b6f9b1e9cd4c4c9007c8c006427bee/users/db9b6a7144e04fb0ac4955b61ed0cacd/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:49.269887 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f061d-d1f7-444a-bade-2cf3ab16524f None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:49.270395 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f061d-d1f7-444a-bade-2cf3ab16524f None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:49.270506 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f061d-d1f7-444a-bade-2cf3ab16524f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:49.297740 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-8a2f061d-d1f7-444a-bade-2cf3ab16524f None None] MFA Rules not processed for user `db9b6a7144e04fb0ac4955b61ed0cacd`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:49.298939 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-8a2f061d-d1f7-444a-bade-2cf3ab16524f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:49.361733 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1489/2971] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:07:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:07:49.395795 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefe398-cfaf-4ce1-be0a-d0e0d33b5dc0 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:49.397512 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-dcefe398-cfaf-4ce1-be0a-d0e0d33b5dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:49.463879 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefe398-cfaf-4ce1-be0a-d0e0d33b5dc0 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:49.467556 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eecd6e-89aa-4a1e-b694-f902975642ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:49.468163 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eecd6e-89aa-4a1e-b694-f902975642ea service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:49.468447 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eecd6e-89aa-4a1e-b694-f902975642ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:49.468718 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eecd6e-89aa-4a1e-b694-f902975642ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:49.477531 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eecd6e-89aa-4a1e-b694-f902975642ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:49.478990 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eecd6e-89aa-4a1e-b694-f902975642ea service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:49.488112 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1483/2972] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:07:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:07:49.504230 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f061d-d1f7-444a-bade-2cf3ab16524f None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:49.567339 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f061d-d1f7-444a-bade-2cf3ab16524f None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:49.571829 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d597a19-fa68-4dc0-817f-8f075f7b1b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:49.572451 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d597a19-fa68-4dc0-817f-8f075f7b1b2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:49.572726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d597a19-fa68-4dc0-817f-8f075f7b1b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:49.572993 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d597a19-fa68-4dc0-817f-8f075f7b1b2c admin admin] PATH_INFO: `/v3/users/db9b6a7144e04fb0ac4955b61ed0cacd` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:49.574156 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d597a19-fa68-4dc0-817f-8f075f7b1b2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=db9b6a7144e04fb0ac4955b61ed0cacd)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:49.576091 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d597a19-fa68-4dc0-817f-8f075f7b1b2c admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:49.622726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-1d597a19-fa68-4dc0-817f-8f075f7b1b2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db9b6a7144e04fb0ac4955b61ed0cacd'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:49.630307 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-1d597a19-fa68-4dc0-817f-8f075f7b1b2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db9b6a7144e04fb0ac4955b61ed0cacd'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:49.637263 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-1d597a19-fa68-4dc0-817f-8f075f7b1b2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db9b6a7144e04fb0ac4955b61ed0cacd'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:49.637902 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-1d597a19-fa68-4dc0-817f-8f075f7b1b2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db9b6a7144e04fb0ac4955b61ed0cacd'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:49.638507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-1d597a19-fa68-4dc0-817f-8f075f7b1b2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db9b6a7144e04fb0ac4955b61ed0cacd'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:49.660296 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1490/2973] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 01:07:49 2022] DELETE /identity/v3/users/db9b6a7144e04fb0ac4955b61ed0cacd => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:49.664829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf40a3-04b5-402e-a964-f606c1ab43ff None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:49.665120 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf40a3-04b5-402e-a964-f606c1ab43ff None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:49.665399 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf40a3-04b5-402e-a964-f606c1ab43ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:49.690942 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-27bf40a3-04b5-402e-a964-f606c1ab43ff None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:49.692603 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-27bf40a3-04b5-402e-a964-f606c1ab43ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:49.737659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:49.766801 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1484/2974] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:07:49.776417 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d597a19-fa68-4dc0-817f-8f075f7b1b2c admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:49.777841 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-1d597a19-fa68-4dc0-817f-8f075f7b1b2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:49.788495 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:49.838068 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d597a19-fa68-4dc0-817f-8f075f7b1b2c admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:49.841728 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb6885-a0b4-4bfa-9aa2-fc2cf883ced5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:49.843204 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb6885-a0b4-4bfa-9aa2-fc2cf883ced5 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:49.843474 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb6885-a0b4-4bfa-9aa2-fc2cf883ced5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:49.843743 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb6885-a0b4-4bfa-9aa2-fc2cf883ced5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:49.853222 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cb6885-a0b4-4bfa-9aa2-fc2cf883ced5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:49.854635 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cb6885-a0b4-4bfa-9aa2-fc2cf883ced5 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:49.864523 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1491/2975] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:07:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:07:50.162902 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf40a3-04b5-402e-a964-f606c1ab43ff None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:50.164396 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-27bf40a3-04b5-402e-a964-f606c1ab43ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:50.177622 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf40a3-04b5-402e-a964-f606c1ab43ff None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:50.182787 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85aefbf-f672-4668-9b6d-3f6e6ff981be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:50.183453 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85aefbf-f672-4668-9b6d-3f6e6ff981be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:50.183734 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85aefbf-f672-4668-9b6d-3f6e6ff981be admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:50.184007 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85aefbf-f672-4668-9b6d-3f6e6ff981be admin admin] PATH_INFO: `/v3/projects/a4b6f9b1e9cd4c4c9007c8c006427bee` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:50.185135 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85aefbf-f672-4668-9b6d-3f6e6ff981be admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4b6f9b1e9cd4c4c9007c8c006427bee)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:50.188228 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85aefbf-f672-4668-9b6d-3f6e6ff981be admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:50.236890 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-d85aefbf-f672-4668-9b6d-3f6e6ff981be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4b6f9b1e9cd4c4c9007c8c006427bee', 'request_id': 'req-d85aefbf-f672-4668-9b6d-3f6e6ff981be'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:50.244376 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-d85aefbf-f672-4668-9b6d-3f6e6ff981be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4b6f9b1e9cd4c4c9007c8c006427bee', 'request_id': 'req-d85aefbf-f672-4668-9b6d-3f6e6ff981be'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:50.251285 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-d85aefbf-f672-4668-9b6d-3f6e6ff981be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4b6f9b1e9cd4c4c9007c8c006427bee', 'request_id': 'req-d85aefbf-f672-4668-9b6d-3f6e6ff981be'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:50.262433 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-d85aefbf-f672-4668-9b6d-3f6e6ff981be admin admin] The token cache is being invalidate because project a4b6f9b1e9cd4c4c9007c8c006427bee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:07:50.262719 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-d85aefbf-f672-4668-9b6d-3f6e6ff981be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:50.263444 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-d85aefbf-f672-4668-9b6d-3f6e6ff981be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:07:50.265762 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1485/2976] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 01:07:50 2022] DELETE /identity/v3/projects/a4b6f9b1e9cd4c4c9007c8c006427bee => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:50.270649 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63273a9b-c782-4db2-9d7e-77ad44c2d8f1 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:50.271128 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63273a9b-c782-4db2-9d7e-77ad44c2d8f1 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:50.271221 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63273a9b-c782-4db2-9d7e-77ad44c2d8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:50.290334 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-63273a9b-c782-4db2-9d7e-77ad44c2d8f1 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:50.291568 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-63273a9b-c782-4db2-9d7e-77ad44c2d8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:50.360538 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1492/2977] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:07:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:07:50.368398 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb39c0-cce1-4ae9-80fd-7ab6e2bb4768 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:50.368687 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb39c0-cce1-4ae9-80fd-7ab6e2bb4768 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:50.368967 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb39c0-cce1-4ae9-80fd-7ab6e2bb4768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:50.391822 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-43cb39c0-cce1-4ae9-80fd-7ab6e2bb4768 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:50.393478 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-43cb39c0-cce1-4ae9-80fd-7ab6e2bb4768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:50.407273 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1486/2978] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:07:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:07:50.411758 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63273a9b-c782-4db2-9d7e-77ad44c2d8f1 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:50.421969 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63273a9b-c782-4db2-9d7e-77ad44c2d8f1 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:50.426520 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f6e5f-1943-4f8f-bcfe-5908d66ba13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:50.427155 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f6e5f-1943-4f8f-bcfe-5908d66ba13c admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:50.427436 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f6e5f-1943-4f8f-bcfe-5908d66ba13c admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:50.427717 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f6e5f-1943-4f8f-bcfe-5908d66ba13c admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:50.428361 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f6e5f-1943-4f8f-bcfe-5908d66ba13c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:07:50.428661 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f6e5f-1943-4f8f-bcfe-5908d66ba13c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:50.430236 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f6e5f-1943-4f8f-bcfe-5908d66ba13c admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:50.442214 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1493/2979] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 01:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 01:07:50.446678 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb39c0-cce1-4ae9-80fd-7ab6e2bb4768 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:50.459549 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb39c0-cce1-4ae9-80fd-7ab6e2bb4768 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:50.465057 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31a56e-b99b-428a-b5f2-b3e4a821dbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:50.465659 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31a56e-b99b-428a-b5f2-b3e4a821dbae admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:50.465939 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31a56e-b99b-428a-b5f2-b3e4a821dbae admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:50.466210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31a56e-b99b-428a-b5f2-b3e4a821dbae admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:50.466872 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d31a56e-b99b-428a-b5f2-b3e4a821dbae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:50.469723 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d31a56e-b99b-428a-b5f2-b3e4a821dbae admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:50.497193 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1487/2980] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 01:07:50 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:07:50.501657 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f6e5f-1943-4f8f-bcfe-5908d66ba13c admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:50.511726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f6e5f-1943-4f8f-bcfe-5908d66ba13c admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:50.567597 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a95eb4-7880-4240-a49e-6cb3158ad5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:50.568206 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a95eb4-7880-4240-a49e-6cb3158ad5e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:50.568484 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a95eb4-7880-4240-a49e-6cb3158ad5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:50.568753 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a95eb4-7880-4240-a49e-6cb3158ad5e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:50.569389 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a95eb4-7880-4240-a49e-6cb3158ad5e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:50.571330 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a95eb4-7880-4240-a49e-6cb3158ad5e3 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:50.599877 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1494/2981] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 01:07:50 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:07:50.603810 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31a56e-b99b-428a-b5f2-b3e4a821dbae admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:50.614672 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31a56e-b99b-428a-b5f2-b3e4a821dbae admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:50.619321 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cacd80-9cb4-426b-bada-b0fe8c2f48db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:50.620037 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cacd80-9cb4-426b-bada-b0fe8c2f48db admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:50.620318 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cacd80-9cb4-426b-bada-b0fe8c2f48db admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:50.620589 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cacd80-9cb4-426b-bada-b0fe8c2f48db admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:50.621227 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cacd80-9cb4-426b-bada-b0fe8c2f48db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:50.622989 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cacd80-9cb4-426b-bada-b0fe8c2f48db admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:50.635649 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1488/2982] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:07:50.640444 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a95eb4-7880-4240-a49e-6cb3158ad5e3 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:50.650938 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a95eb4-7880-4240-a49e-6cb3158ad5e3 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:50.656570 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7f3ed-ac60-4a09-881a-b1e411f71fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:50.657175 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7f3ed-ac60-4a09-881a-b1e411f71fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:50.657456 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7f3ed-ac60-4a09-881a-b1e411f71fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:50.657731 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7f3ed-ac60-4a09-881a-b1e411f71fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:50.658374 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7f3ed-ac60-4a09-881a-b1e411f71fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:50.660014 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7f3ed-ac60-4a09-881a-b1e411f71fa0 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:50.670792 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1495/2983] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:07:50.674957 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cacd80-9cb4-426b-bada-b0fe8c2f48db admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:50.684850 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cacd80-9cb4-426b-bada-b0fe8c2f48db admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:50.690363 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66ff11-2cc7-416f-944d-103718e10851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:50.691010 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66ff11-2cc7-416f-944d-103718e10851 admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:50.691310 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66ff11-2cc7-416f-944d-103718e10851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:50.691582 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66ff11-2cc7-416f-944d-103718e10851 admin admin] PATH_INFO: `/v3/projects/3159109326b8415aa46f8df9b2d379be/users/520d38cbc6694ae3aaf7edd5efdd9ea3/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:50.707997 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e66ff11-2cc7-416f-944d-103718e10851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3159109326b8415aa46f8df9b2d379be, user_id=520d38cbc6694ae3aaf7edd5efdd9ea3, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:50.710619 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e66ff11-2cc7-416f-944d-103718e10851 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:50.723695 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1489/2984] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 01:07:50 2022] PUT /identity/v3/projects/3159109326b8415aa46f8df9b2d379be/users/520d38cbc6694ae3aaf7edd5efdd9ea3/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:07:50.734159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e21b3-1d8d-4203-b213-c34376ef64ab None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:50.734501 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e21b3-1d8d-4203-b213-c34376ef64ab None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:50.734971 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e21b3-1d8d-4203-b213-c34376ef64ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:50.762443 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-655e21b3-1d8d-4203-b213-c34376ef64ab None None] MFA Rules not processed for user `520d38cbc6694ae3aaf7edd5efdd9ea3`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:07:50.763574 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-655e21b3-1d8d-4203-b213-c34376ef64ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:50.768262 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:50.830003 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1496/2985] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:07:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:07:50.841663 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66ff11-2cc7-416f-944d-103718e10851 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:07:50.843946 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-9e66ff11-2cc7-416f-944d-103718e10851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:07:50.899808 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66ff11-2cc7-416f-944d-103718e10851 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:07:50.903434 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac5bbd-4885-4b2f-a52d-6b35082edc75 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55714a21c1cb4f2b949603230243ed02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:07:50.904033 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac5bbd-4885-4b2f-a52d-6b35082edc75 service glance] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:07:50.904826 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac5bbd-4885-4b2f-a52d-6b35082edc75 service glance] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:07:50.905115 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac5bbd-4885-4b2f-a52d-6b35082edc75 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:07:50.913278 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac5bbd-4885-4b2f-a52d-6b35082edc75 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:07:50.914689 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac5bbd-4885-4b2f-a52d-6b35082edc75 service glance] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:07:50.919162 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:07:50.924699 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1490/2986] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:07:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:08:00.826261 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=25 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:08:28.190322 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e21b3-1d8d-4203-b213-c34376ef64ab None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:08:28.270341 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e21b3-1d8d-4203-b213-c34376ef64ab None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:08:28.274841 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b2b4f-4da3-4209-b8a8-5df1fb60ed8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:08:28.275491 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b2b4f-4da3-4209-b8a8-5df1fb60ed8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:28.275770 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b2b4f-4da3-4209-b8a8-5df1fb60ed8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:28.276043 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b2b4f-4da3-4209-b8a8-5df1fb60ed8d admin admin] PATH_INFO: `/v3/users/520d38cbc6694ae3aaf7edd5efdd9ea3` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:28.277150 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b2b4f-4da3-4209-b8a8-5df1fb60ed8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=520d38cbc6694ae3aaf7edd5efdd9ea3)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:08:28.279087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b2b4f-4da3-4209-b8a8-5df1fb60ed8d admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:08:28.326100 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3a6b2b4f-4da3-4209-b8a8-5df1fb60ed8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '520d38cbc6694ae3aaf7edd5efdd9ea3'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:08:28.333988 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3a6b2b4f-4da3-4209-b8a8-5df1fb60ed8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '520d38cbc6694ae3aaf7edd5efdd9ea3'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:08:28.341134 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3a6b2b4f-4da3-4209-b8a8-5df1fb60ed8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '520d38cbc6694ae3aaf7edd5efdd9ea3'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:08:28.341779 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3a6b2b4f-4da3-4209-b8a8-5df1fb60ed8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '520d38cbc6694ae3aaf7edd5efdd9ea3'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:08:28.342395 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-3a6b2b4f-4da3-4209-b8a8-5df1fb60ed8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '520d38cbc6694ae3aaf7edd5efdd9ea3'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:08:28.362057 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1497/2987] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 01:08:28 2022] DELETE /identity/v3/users/520d38cbc6694ae3aaf7edd5efdd9ea3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:08:28.366665 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09315e-7fd0-48d6-a06c-345b7ea1c1ca None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:28.367187 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09315e-7fd0-48d6-a06c-345b7ea1c1ca None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:28.367317 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09315e-7fd0-48d6-a06c-345b7ea1c1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:28.407127 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-4d09315e-7fd0-48d6-a06c-345b7ea1c1ca None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:08:28.408332 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-4d09315e-7fd0-48d6-a06c-345b7ea1c1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:08:28.470805 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1491/2988] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:08:28.481117 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b2b4f-4da3-4209-b8a8-5df1fb60ed8d admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:08:28.482588 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-3a6b2b4f-4da3-4209-b8a8-5df1fb60ed8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:08:28.539203 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b2b4f-4da3-4209-b8a8-5df1fb60ed8d admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:08:28.542859 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a71e0b-69a1-4108-936a-59495b95ca7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:08:28.543489 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a71e0b-69a1-4108-936a-59495b95ca7d service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:28.543766 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a71e0b-69a1-4108-936a-59495b95ca7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:28.544036 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a71e0b-69a1-4108-936a-59495b95ca7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:28.553680 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a71e0b-69a1-4108-936a-59495b95ca7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:08:28.555128 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a71e0b-69a1-4108-936a-59495b95ca7d service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:08:28.565120 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1498/2989] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:08:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:08:28.838466 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09315e-7fd0-48d6-a06c-345b7ea1c1ca None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:08:28.839915 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-4d09315e-7fd0-48d6-a06c-345b7ea1c1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:08:28.853319 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09315e-7fd0-48d6-a06c-345b7ea1c1ca None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:08:28.858181 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b443c1-2390-4cd5-a9e8-c7b071f583db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:08:28.858806 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b443c1-2390-4cd5-a9e8-c7b071f583db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:28.859103 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b443c1-2390-4cd5-a9e8-c7b071f583db admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:28.859380 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b443c1-2390-4cd5-a9e8-c7b071f583db admin admin] PATH_INFO: `/v3/projects/3159109326b8415aa46f8df9b2d379be` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:28.861129 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b443c1-2390-4cd5-a9e8-c7b071f583db admin admin] RBAC: Authorizing `identity:delete_project(project_id=3159109326b8415aa46f8df9b2d379be)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:08:28.863052 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b443c1-2390-4cd5-a9e8-c7b071f583db admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:08:28.910767 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e7b443c1-2390-4cd5-a9e8-c7b071f583db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3159109326b8415aa46f8df9b2d379be', 'request_id': 'req-e7b443c1-2390-4cd5-a9e8-c7b071f583db'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:08:28.918257 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e7b443c1-2390-4cd5-a9e8-c7b071f583db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3159109326b8415aa46f8df9b2d379be', 'request_id': 'req-e7b443c1-2390-4cd5-a9e8-c7b071f583db'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:08:28.924395 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e7b443c1-2390-4cd5-a9e8-c7b071f583db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3159109326b8415aa46f8df9b2d379be', 'request_id': 'req-e7b443c1-2390-4cd5-a9e8-c7b071f583db'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:08:28.935180 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e7b443c1-2390-4cd5-a9e8-c7b071f583db admin admin] The token cache is being invalidate because project 3159109326b8415aa46f8df9b2d379be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:08:28.935455 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e7b443c1-2390-4cd5-a9e8-c7b071f583db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:08:28.936085 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-e7b443c1-2390-4cd5-a9e8-c7b071f583db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:08:28.937560 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1492/2990] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 01:08:28 2022] DELETE /identity/v3/projects/3159109326b8415aa46f8df9b2d379be => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:08:28.942530 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8ed81-8f9e-4f43-9546-196b029da6d6 None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:28.942814 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8ed81-8f9e-4f43-9546-196b029da6d6 None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:28.943112 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8ed81-8f9e-4f43-9546-196b029da6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:28.950582 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:08:28.998781 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-44e8ed81-8f9e-4f43-9546-196b029da6d6 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:08:29.000019 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-44e8ed81-8f9e-4f43-9546-196b029da6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:08:29.059101 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1499/2991] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:08:29.067311 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cfcb4-5492-4e44-ab64-56d9c2157657 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:29.067603 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cfcb4-5492-4e44-ab64-56d9c2157657 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:29.067883 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cfcb4-5492-4e44-ab64-56d9c2157657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:29.087952 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-761cfcb4-5492-4e44-ab64-56d9c2157657 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:08:29.089150 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-761cfcb4-5492-4e44-ab64-56d9c2157657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:08:29.098116 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1493/2992] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:08:29.102286 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8ed81-8f9e-4f43-9546-196b029da6d6 None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:08:29.112386 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8ed81-8f9e-4f43-9546-196b029da6d6 None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:08:29.116964 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2450fc-1b49-43ef-85a5-0b685b6a5eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:08:29.117559 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2450fc-1b49-43ef-85a5-0b685b6a5eaa admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:29.117837 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2450fc-1b49-43ef-85a5-0b685b6a5eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:29.118105 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2450fc-1b49-43ef-85a5-0b685b6a5eaa admin admin] PATH_INFO: `/v3/domains` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:29.118754 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2450fc-1b49-43ef-85a5-0b685b6a5eaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81472) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 01:08:29.119087 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2450fc-1b49-43ef-85a5-0b685b6a5eaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:08:29.120687 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2450fc-1b49-43ef-85a5-0b685b6a5eaa admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:08:29.132922 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1500/2993] 10.222.0.67 () {58 vars in 1153 bytes} [Wed Aug 31 01:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 01:08:29.137210 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cfcb4-5492-4e44-ab64-56d9c2157657 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:08:29.147269 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cfcb4-5492-4e44-ab64-56d9c2157657 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:08:29.151929 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5875d1b8-9d6d-40a9-9f0b-8f1d55dfe047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:08:29.152554 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875d1b8-9d6d-40a9-9f0b-8f1d55dfe047 admin admin] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:29.152829 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875d1b8-9d6d-40a9-9f0b-8f1d55dfe047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:29.153099 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875d1b8-9d6d-40a9-9f0b-8f1d55dfe047 admin admin] PATH_INFO: `/v3/projects` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:29.153721 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5875d1b8-9d6d-40a9-9f0b-8f1d55dfe047 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:08:29.155640 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5875d1b8-9d6d-40a9-9f0b-8f1d55dfe047 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:08:29.192372 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1494/2994] 10.222.0.67 () {60 vars in 1153 bytes} [Wed Aug 31 01:08:29 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:08:29.196507 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2450fc-1b49-43ef-85a5-0b685b6a5eaa admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:08:29.207435 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2450fc-1b49-43ef-85a5-0b685b6a5eaa admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:08:29.263620 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448ce23-7806-412c-ad32-f8f5f0b5ba57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:08:29.264182 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448ce23-7806-412c-ad32-f8f5f0b5ba57 admin admin] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:29.264461 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448ce23-7806-412c-ad32-f8f5f0b5ba57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:29.264729 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448ce23-7806-412c-ad32-f8f5f0b5ba57 admin admin] PATH_INFO: `/v3/users` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:29.265360 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b448ce23-7806-412c-ad32-f8f5f0b5ba57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:08:29.267333 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b448ce23-7806-412c-ad32-f8f5f0b5ba57 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:08:29.288242 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1501/2995] 10.222.0.67 () {60 vars in 1144 bytes} [Wed Aug 31 01:08:29 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 01:08:29.292074 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5875d1b8-9d6d-40a9-9f0b-8f1d55dfe047 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:08:29.302938 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5875d1b8-9d6d-40a9-9f0b-8f1d55dfe047 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:08:29.307588 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16023baf-5d2c-47b5-a2d1-9dca9c035290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:08:29.308185 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16023baf-5d2c-47b5-a2d1-9dca9c035290 admin admin] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:29.308468 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16023baf-5d2c-47b5-a2d1-9dca9c035290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:29.308736 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16023baf-5d2c-47b5-a2d1-9dca9c035290 admin admin] PATH_INFO: `/v3/roles` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:29.309367 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16023baf-5d2c-47b5-a2d1-9dca9c035290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:08:29.310954 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16023baf-5d2c-47b5-a2d1-9dca9c035290 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:08:29.321634 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1495/2996] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:08:29.325477 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448ce23-7806-412c-ad32-f8f5f0b5ba57 admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:08:29.335946 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448ce23-7806-412c-ad32-f8f5f0b5ba57 admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:08:29.340576 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a5ced-1899-43ae-9b9d-a23198f8ea86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:08:29.341163 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a5ced-1899-43ae-9b9d-a23198f8ea86 admin admin] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:29.341444 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a5ced-1899-43ae-9b9d-a23198f8ea86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:29.341714 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a5ced-1899-43ae-9b9d-a23198f8ea86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:29.342345 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94a5ced-1899-43ae-9b9d-a23198f8ea86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:08:29.343974 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94a5ced-1899-43ae-9b9d-a23198f8ea86 admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:08:29.355219 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1502/2997] 10.222.0.67 () {58 vars in 1122 bytes} [Wed Aug 31 01:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 01:08:29.359426 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16023baf-5d2c-47b5-a2d1-9dca9c035290 admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:08:29.372032 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16023baf-5d2c-47b5-a2d1-9dca9c035290 admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:08:29.377696 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110b341-94a3-4796-83dd-874bb4bfefca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:08:29.378358 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110b341-94a3-4796-83dd-874bb4bfefca admin admin] REQUEST_METHOD: `PUT` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:29.378642 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110b341-94a3-4796-83dd-874bb4bfefca admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:29.378919 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110b341-94a3-4796-83dd-874bb4bfefca admin admin] PATH_INFO: `/v3/projects/cc892964a9864db791b9ca7bcb7d5e45/users/665789ca31d8475caef166cf2f09646f/roles/cc8bd97c7b9745d1a52746650a2e49b9` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:29.394584 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110b341-94a3-4796-83dd-874bb4bfefca admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc892964a9864db791b9ca7bcb7d5e45, user_id=665789ca31d8475caef166cf2f09646f, role_id=cc8bd97c7b9745d1a52746650a2e49b9)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:08:29.397241 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110b341-94a3-4796-83dd-874bb4bfefca admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:08:29.399468 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:08:29.412935 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1496/2998] 10.222.0.67 () {60 vars in 1483 bytes} [Wed Aug 31 01:08:29 2022] PUT /identity/v3/projects/cc892964a9864db791b9ca7bcb7d5e45/users/665789ca31d8475caef166cf2f09646f/roles/cc8bd97c7b9745d1a52746650a2e49b9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:08:29.423317 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a835a-43c8-443c-8fe4-7e13a967c44b None None] REQUEST_METHOD: `POST` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:29.423596 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a835a-43c8-443c-8fe4-7e13a967c44b None None] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:29.423863 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a835a-43c8-443c-8fe4-7e13a967c44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:29.450975 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.auth.core [None req-4f3a835a-43c8-443c-8fe4-7e13a967c44b None None] MFA Rules not processed for user `665789ca31d8475caef166cf2f09646f`. Rule list: `[]` (Enabled: `True`). {{(pid=81472) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:08:29.452225 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-4f3a835a-43c8-443c-8fe4-7e13a967c44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:08:29.485425 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:08:29.516963 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1503/2999] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:08:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 01:08:29.527987 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110b341-94a3-4796-83dd-874bb4bfefca admin admin] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:08:29.529650 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-e110b341-94a3-4796-83dd-874bb4bfefca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:08:29.598777 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110b341-94a3-4796-83dd-874bb4bfefca admin admin] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:08:29.602430 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e9d04-1de2-48f8-b04c-2394e67895b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e42dfb7d22e4435889cb91327db5477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:08:29.603059 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e9d04-1de2-48f8-b04c-2394e67895b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:29.603342 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e9d04-1de2-48f8-b04c-2394e67895b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:29.603616 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e9d04-1de2-48f8-b04c-2394e67895b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:29.612419 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0e9d04-1de2-48f8-b04c-2394e67895b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:08:29.613841 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0e9d04-1de2-48f8-b04c-2394e67895b4 service cinder] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:08:29.622854 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1497/3000] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:08:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:08:29.647694 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a835a-43c8-443c-8fe4-7e13a967c44b None None] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:08:29.714949 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a835a-43c8-443c-8fe4-7e13a967c44b None None] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:08:29.720067 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750f83b-d37d-4fd3-8c4d-75e9ee8659df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:08:29.720770 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750f83b-d37d-4fd3-8c4d-75e9ee8659df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:29.721042 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750f83b-d37d-4fd3-8c4d-75e9ee8659df admin admin] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:29.721306 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750f83b-d37d-4fd3-8c4d-75e9ee8659df admin admin] PATH_INFO: `/v3/users/665789ca31d8475caef166cf2f09646f` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:29.722693 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750f83b-d37d-4fd3-8c4d-75e9ee8659df admin admin] RBAC: Authorizing `identity:delete_user(user_id=665789ca31d8475caef166cf2f09646f)` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:08:29.724633 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750f83b-d37d-4fd3-8c4d-75e9ee8659df admin admin] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:08:29.773216 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-e750f83b-d37d-4fd3-8c4d-75e9ee8659df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '665789ca31d8475caef166cf2f09646f'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:08:29.781236 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-e750f83b-d37d-4fd3-8c4d-75e9ee8659df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '665789ca31d8475caef166cf2f09646f'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:08:29.789623 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-e750f83b-d37d-4fd3-8c4d-75e9ee8659df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '665789ca31d8475caef166cf2f09646f'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:08:29.790305 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-e750f83b-d37d-4fd3-8c4d-75e9ee8659df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '665789ca31d8475caef166cf2f09646f'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:08:29.790903 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.notifications [None req-e750f83b-d37d-4fd3-8c4d-75e9ee8659df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '665789ca31d8475caef166cf2f09646f'} {{(pid=81472) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:08:29.808665 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1504/3001] 10.222.0.67 () {58 vars in 1224 bytes} [Wed Aug 31 01:08:29 2022] DELETE /identity/v3/users/665789ca31d8475caef166cf2f09646f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:08:29.814165 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65616941-8ab1-4c80-9bc0-e3aef855c721 None None] REQUEST_METHOD: `POST` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:29.814457 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65616941-8ab1-4c80-9bc0-e3aef855c721 None None] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:29.814732 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65616941-8ab1-4c80-9bc0-e3aef855c721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:29.833611 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.auth.core [None req-65616941-8ab1-4c80-9bc0-e3aef855c721 None None] MFA Rules not processed for user `19ddd6d02b884bcab507cb5ed1181266`. Rule list: `[]` (Enabled: `True`). {{(pid=81471) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 01:08:29.834798 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-65616941-8ab1-4c80-9bc0-e3aef855c721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:08:29.895366 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1498/3002] 10.222.0.67 () {58 vars in 958 bytes} [Wed Aug 31 01:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 01:08:29.905158 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750f83b-d37d-4fd3-8c4d-75e9ee8659df admin admin] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:08:29.906554 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-e750f83b-d37d-4fd3-8c4d-75e9ee8659df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:08:29.979649 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750f83b-d37d-4fd3-8c4d-75e9ee8659df admin admin] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:08:29.983514 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c195e7a-3134-4b96-92c1-a6f13c7e64b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05c0d9554cf94c41a2cd709cab378e53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:08:29.984123 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c195e7a-3134-4b96-92c1-a6f13c7e64b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:29.984400 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c195e7a-3134-4b96-92c1-a6f13c7e64b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:29.984669 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c195e7a-3134-4b96-92c1-a6f13c7e64b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:29.996231 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c195e7a-3134-4b96-92c1-a6f13c7e64b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:08:29.997726 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c195e7a-3134-4b96-92c1-a6f13c7e64b5 service neutron] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:08:30.003160 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:08:30.010998 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1505/3003] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:08:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:08:30.278318 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65616941-8ab1-4c80-9bc0-e3aef855c721 None None] Authenticating user token {{(pid=81471) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:08:30.279724 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.fernet_utils [None req-65616941-8ab1-4c80-9bc0-e3aef855c721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81471) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:08:30.292779 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65616941-8ab1-4c80-9bc0-e3aef855c721 None None] Validating token access rules against request {{(pid=81471) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:08:30.297508 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fddc5ea-b276-4735-b1df-d35d005c9662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ddd6d02b884bcab507cb5ed1181266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1509e8ef0c6042559698c40a7a07a646', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81471) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:08:30.299069 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddc5ea-b276-4735-b1df-d35d005c9662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:08:30.299496 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddc5ea-b276-4735-b1df-d35d005c9662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:08:30.310853 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddc5ea-b276-4735-b1df-d35d005c9662 admin admin] PATH_INFO: `/v3/projects/cc892964a9864db791b9ca7bcb7d5e45` {{(pid=81471) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:08:30.312526 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fddc5ea-b276-4735-b1df-d35d005c9662 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc892964a9864db791b9ca7bcb7d5e45)` {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:08:30.315276 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fddc5ea-b276-4735-b1df-d35d005c9662 admin admin] RBAC: Authorization granted {{(pid=81471) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:08:30.381559 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-6fddc5ea-b276-4735-b1df-d35d005c9662 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc892964a9864db791b9ca7bcb7d5e45', 'request_id': 'req-6fddc5ea-b276-4735-b1df-d35d005c9662'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:08:30.389749 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-6fddc5ea-b276-4735-b1df-d35d005c9662 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc892964a9864db791b9ca7bcb7d5e45', 'request_id': 'req-6fddc5ea-b276-4735-b1df-d35d005c9662'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:08:30.395893 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-6fddc5ea-b276-4735-b1df-d35d005c9662 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc892964a9864db791b9ca7bcb7d5e45', 'request_id': 'req-6fddc5ea-b276-4735-b1df-d35d005c9662'} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:08:30.407652 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-6fddc5ea-b276-4735-b1df-d35d005c9662 admin admin] The token cache is being invalidate because project cc892964a9864db791b9ca7bcb7d5e45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81471) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 01:08:30.407943 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-6fddc5ea-b276-4735-b1df-d35d005c9662 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:08:30.408577 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG keystone.notifications [None req-6fddc5ea-b276-4735-b1df-d35d005c9662 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81471) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 01:08:30.410026 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: [pid: 81471|app: 0|req: 1499/3004] 10.222.0.67 () {58 vars in 1233 bytes} [Wed Aug 31 01:08:30 2022] DELETE /identity/v3/projects/cc892964a9864db791b9ca7bcb7d5e45 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 01:08:40.013778 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=1 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:08:40.403993 dv-cinder-855060-1-iscsi devstack@keystone.service[81471]: DEBUG dbcounter [-] [81471] Writing DB stats keystone:INSERT=2,keystone:SELECT=87,keystone:DELETE=7 {{(pid=81471) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 01:10:23.915796 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c195e7a-3134-4b96-92c1-a6f13c7e64b5 service neutron] Authenticating user token {{(pid=81472) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 01:10:23.917192 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-8c195e7a-3134-4b96-92c1-a6f13c7e64b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:10:24.013160 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c195e7a-3134-4b96-92c1-a6f13c7e64b5 service neutron] Validating token access rules against request {{(pid=81472) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 01:10:24.074159 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b767c5-29a5-4b06-809c-b75ea4a981a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdacdcb2451742938164fdd44cdbaebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd1e6daca4264a0cbc11a9160d1213eb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81472) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 01:10:24.074763 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b767c5-29a5-4b06-809c-b75ea4a981a8 service placement] REQUEST_METHOD: `GET` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 01:10:24.075181 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b767c5-29a5-4b06-809c-b75ea4a981a8 service placement] SCRIPT_NAME: `/identity` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 01:10:24.075488 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b767c5-29a5-4b06-809c-b75ea4a981a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81472) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 01:10:24.077215 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.fernet_utils [None req-b1b767c5-29a5-4b06-809c-b75ea4a981a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81472) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 01:10:24.089541 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b767c5-29a5-4b06-809c-b75ea4a981a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 01:10:24.091148 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b767c5-29a5-4b06-809c-b75ea4a981a8 service placement] RBAC: Authorization granted {{(pid=81472) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 01:10:24.101084 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: [pid: 81472|app: 0|req: 1506/3005] 10.222.0.67 () {60 vars in 1365 bytes} [Wed Aug 31 01:10:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 01:10:34.096656 dv-cinder-855060-1-iscsi devstack@keystone.service[81472]: DEBUG dbcounter [-] [81472] Writing DB stats keystone:SELECT=67 {{(pid=81472) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}